Calibrating a tape deck?

Er, have you tried getting a service manual from Fostex?

I hope you can do it without a scope..

Hows it sounding as it is?

Have you tried waving burning incense sticks above it?

:)

The following is taken from the FAQ for R.A.P.

• Q4.1 - What does it mean to "align" a tape machine?
• There are a number of standard adjustments on any analog tape machine, which can roughly be broken up into mechanical and electronic adjustments. The mechanical adjustments include the head position (height, skew, and azimuth), and sometimes tape speed. Incorrect head height will result in poor S/N and leakage between channels, because the tracks on the head do not match up exactly with those on the tape. Incorrect tape skew will result in level differences between channels and uneven head wear, because there is more pressure on the top of the head than the bottom (or vice versa). Incorrect azimuth will result in loss of high frequency response and strange skewing of the stereo image. Tape speed error will result in tonal shifts. Although on many machines with capstan speed controlled by crystal or line frequency, it is not adjustable.
• Electronic adjustments include level and bias adjustments for each channel. Some machines may have bias frequency adjustments, equalization adjustments for playback and record emphasis, pre-distortion adjustments, and a varied bevy of adjustments for noise reduction systems.
• Alignment is relatively simple, and the same general method applies from the smallest cassette deck to the largest multi-track machine. First, put a test tape on the machine. Use a real reference tape, from the manufacturer, from MRL, or a similarly legitimate lab.
• DO NOT EVER use a home-brew test tape that was recorded on a "known good" machine. You will regret it someday. Spend the money and get a real test tape (and not one of the flaky ones from RCA).

1. Speed adjustment (if necessary). Play back a 1 KHz reference tone and, using a frequency counter, adjust the tape speed for proper frequency output. There are strobe tapes available for this as well, but with cheap frequency counters available, this method is much easier.

2. Head height and skew adjustments. Better see your machine's manual on this one, because I have seen a variety of ways of doing this. If in doubt,leave it alone.

3. Azimuth adjustment. I find the easiest way to do this is to take the left and right outputs and connect them to the X and Y inputs of an oscilloscope, and play back a 10 kHz reference
tone, while adjusting the azimuth until a perfectly-diagonal line appears. You can do this by ear if you are desperate, but I strongly recommend the Lissajous method, which is faster and more accurate. On multi-track decks, use the two tracks as close as possible to the opposite edges of the tape. Now you have the playback head azimuth set... put a 10 kHz source into the record input, with a blank tape on the machine, and adjust the azimuth of the record head for the proper diagonal line from the playback output. If the alignment is very far off to begin with, it might be easier to do these steps first with a 1 kHz tone to get the azimuth adjustments close, and then switch to a 10 kHz tone for fine adjustment.

4. Playback level adjustment. Adjust the playback meters to show 0 dB when a 1 Khz 0 dB signal is played back. (This assumes that your reference tape has the same fluxivity as the tape you're going to be using.... if you want to work at a hotter level, you may want to adjust the meters to show a level lower than 0 dB or get a different tape... and there should be a set of markings on the tape to tell you what levels to set for what fluxivities).

5. Playback eq adjustment (if necessary). This is a case of playing back various test tones at different frequencies, and adjusting the response curve of the deck to produce a flat output. You can also do this by playing back white noise and using a third-octave spectrum analyzer of great accuracy to adjust for flat response. Again, this is one to check your deck's manual for, because the actual adjustments vary from one machine to another, and you will want to use the test tape once again. There is a good explanation of various choices in tape equalization at http://www.flash.net/~mrltapes/choo&u.pdf. Most commonly, though, there is a low end response setting that is adjusted around 100 Hz and a high end response setting that is adjusted around 15 kHz. It's often a good idea to look at the various levels of a stepped tone sequence, too, just to make sure the playback levels really are flat.

Now it's time to take the reference tape off the machine, and put a blank tape OF THE TYPE YOU ARE GOING TO BE USING on the machine. Set up the machine or external meters to look at the repro levels coming off the tape.

6. Bias adjustment. There are a lot of ways to do this. The common way to do this is to record a high frequency to the tape, and adjust the bias until the signal peaks, then increase the level of the bias until the signal coming off the tape drops by a little bit. That little bit is called the overbias, and you should check on the data sheet of the tape you are using and see what level of overbias is recommended for that tape. Usually this is done at 10 kHz for 15 ips tape.

Other folks will use a 1 KHz tone and adjust for a peak with no overbias, and this is the method recommended in the manual for many older tape machines. However, it's not optimal for most of the newer tape formulations. Some other folks record a low frequency tone and adjust for lowest modulation noise, a method which also has adherents. Previous copies of this FAQ recommended adjusting for flattest response, a method I no longer recommend.

7. Record eq adjustment (if necessary). How this is done (and whether you want to do it after biasing the tape) depends a lot on your deck. The idea is to adjust the controls, again usually one for low frequencies and one for highs, so that the playback output tracks the record input. You have already adjusted the playback eq so that it plays an alignment tape with uniform output, now you adjust the record eq so that the record section prints the same flat response on to the tape. It's important to reduce the record level when doing these adjustments, so that the high frequencies don't overload the tape.

8. Record level adjustment. This is really nothing more than calibrating the record VU meter. The most common way to do this is to just adjust so that the level coming off the tape is the same as the level going on the tape. Adjust the meter so you get the same level set to repro and to input monitor modes. Other folks prefer to set the levels for a particular distortion level, either by using a 1 KHz tone and setting so the onset of audible distortion is at +3 dB or using a distortion analyzer, then going back and resetting the playback EQ so that the levels match (thereby ignoring the level set from the reference tape).

At this point, you will be pretty much set. Whether you want to do this all on a regular basis is a good question. You should definitely go through the complete procedure if you ever change brands of tape. Checking the mechanical parameters on a regular basis is a good idea with some decks (like the Ampex 350), which tend to drift. Clean your heads every time you put a new reel on, and demagnetize regularly.