Check you state laws for employment contract enforcement. Usually consultancies have contracts with Prime Vendors. If your prime vendor isnt complaining then its end of story. As far as the Client goes, client didnt hire you from consultancy, he hired you from Prime Vendor.

All these desi consultancies prey on ppl with agreements. If push comes to shove, have your manager end your contract for a week, then re hire.

Non Compete agreement is usually a clause that prevents you from working for a rival company. If you can post a redacted version of your agreement that you are concerned about, may be law savvy folks can chime in. I doubt there is any kind of agreement that lets your client in conflict with your consultancy.
You were probably on a contract to hire position and consultancy would have no right to stop you from getting a FTE with the client (at least from client's perspective).
Having said that, if you have signed an agreement with your consultancy, they might drag you (not your client) to court. I have heard of few folks being sued and also heard of such results ending in settlement.
Talk politely with your consultancy, thank them for their help you received when you really needed it and tell them that you will refer them people and request them to let you take this great opportunity.
If that doesn't work out, you have 2 options:
1) Find another job on your own (if you can/ want)
2) Still take FTE and if they come after you, hire a lawyer and settle the case. They usually start out with cease and desist letter without any intention of going to the court. The person I once knew dug a $5000 (one time thing, including lawyer's fee) hole in a 90K salary from what I heard.

I was able to talk my consultancy out of my a$$ when I was in your shoe few years ago.
Thank You