Forget the jargon -- the important thing is to reach a meeting of the minds with the reporter about the conditions imposed on the interview or information, BEFORE you give the interview or share the information. The reporter needs a chance to agree or disagree with the proposed terms and you both need to have the same understanding so there are no surprises later. A surprise will not be in your favor so it's in your interest to get clarity.
The general rule in American print journalism is that a reporter always has to identify themselves as a reporter, and from that point on, you are "on the record" unless you reach an agreement otherwise. That means everything can be attributed to you by name.
Regarding the terms -- Generally speaking, information delivered "on background" might be directly quoted (meaning a verbatim quotation inside quotation marks) and attributed to "a source close to FooCorp" or "a FooCorp executive." Information delivered "on deep background" might be paraphrased and attributed to "people familiar with the matter" (this is a favorite Wall Street Journal phrase) or "according to various estimates" or simply inserted in the article without attribution.
Information that's "off the record" isn't supposed to appear in the article at all unless given by another source -- but without a firm understanding otherwise, the reporter may use the "off the record" information to try to pump other sources to confirm it. This could easily make it known to the other sources that you were the original source of the information.
Every reporter and publication has a slightly different understanding of what is and isn't permissible with "background," "deep background," "not for attribution" and "off the record" material, so again, better to get a real meeting of the minds than to use the jargon and be surprised later.