Interviewer(s): Kevin M. Rusnak, Tim Farrell
Interviewee(s): Robert L. Carlton

Kevin M. Rusnak: Today is March 29th, 2001. This interview with Bob Carlton is being conducted in the offices of the Signal Corporation in Houston, Texas, for the Johnson Space Center Oral History Project. The interviewer is Kevin Rusnak, assisted by Sandra Johnson and Tim Farrell.\n\n I'd like to thank you for taking the time out this afternoon to participate in the project. If we could begin, tell us some about your growing up and maybe some of the interests or experiences you had that led you into engineering and in aviation.

Robert L. Carlton: I guess, like most of us, as a young man I didn't have much of a plan about where I was going. I ended up in aerospace as a sort of an accident of a combination of events that sort of led me that way. If I pick up about where I was started into the branch that brought me into aerospace was a completely totally unplanned thing. As a teenager and growing up as a young adult, I aspired to be a businessman, and I was still in high school when I joined the [Alabama] National Guard. All the fellow students went in the Guard. That was a weekend warrior game. You got a little extra money out of it, and there's always a glamour to that. So that's what sucked me in as something that you just at the time would have never dreamed where it would lead, but that's what led me to the aerospace world, unbeknownst to me at the time.\n\n The reason it led to the aerospace world is when the Korean War came along, the National Guard got called up, and as it was being called up, I decided I'd rather be in the Air Force. So I joined the Air Force, and in doing that, that earlier decision for the weekend fun, I was at a crossroads and didn't know it and ended up in the Air Force, which really was the stepping stone into the aerospace world. When you get involved in aircraft work, and I think the military sort of grows you up in a hurry, at that point becoming involved in aircraft for four years in the Air Force, well, that sort of sets your mind-set to where you think aircraft, and then as aerospace come along, you think aerospace.\n\n I wasn't in the Air Force very long before I realized the lack of an education was a big handicap. So while in the Air Force, I woke up to the need to have an education and resolved to do that. After getting out of the Air Force, I went on to Auburn [University, Alabama].\n\n While I was in the Air Force, I worked on the flight line on aircraft, and I can see now a great parallel in what I was doing there to what I ended up doing in flight control. The aircraft mechanic's job was to find problems and fix problems and repair problems. So when an airplane came in, the pilot came down with a big long list of complaints, this went wrong, that went wrong, and you went out on the flight line then and tried to find where the problem was. So you started out with the knowledge of how your system worked, air-conditioning systems in the airplane and powerplants and the guidance and control. They mostly had manual control in those days. You had fire control systems on the guns.\n\n In all of those, I could see, really, a one-for-one parallel to what we did in mission control. You [had] problems occurring, you fixed them. That was just a piece of mission control. In mission control there's more to it than that. You not only have a problem and you try to fix it, but you fix it with a view in mind of trying to maximize the game downstream. You're going to try to make it limp along until you get to a point where you can totally fix it. You rarely ever can fix something on orbit, but on the ground, it was just repair and fix.\n\n There was another difference. On the ground with aircraft, it worked according to spec, and if it didn't, you pulled the boxes out and replaced them. When you're in orbit, you don't have that option as a general rule. Now, what you try to do is make it limp along and finish as much of the flight as you can, as long as it can be done safely. But nevertheless, the diagnostic steps you went through to ascertain something's wrong, do I have a leak in this system? Is there a short in this system? Those diagnostic steps were the same. So that whole four years in the Air Force, in reflecting back on it, I think really couldn't have been a better primer and preparatory step to subsequently working in flight operations.\n\n After leaving the Air Force, I went to Auburn and got a degree as a mechanical engineer. For me, that was a great struggle. I didn't mention it earlier, but I quit school. Ninth grade was the last grade of school that I finished. I started into the tenth, and about three months into it, I decided I could become rich as a businessman and didn't need an education. So I had an especially hard time in the first year or two years of college. That was a pretty good handicap. But it all worked out, and I ended up with a mechanical engineering degree from Auburn.\n\n When I got out of Auburn, it was just sort of natural to go back to the aerospace industry. I ended up back at General Dynamics [Corp.], Convair in the early days it was called, and then it changed its name to General Dynamics. At the time I came back to General Dynamics, they were working on the B-58 [Hustler] aircraft, which is a four-engine jet bomber. It was put in service for a short period of time and then retired.\n\n At the time I went to General Dynamics, they were just on the verge of finishing up the flight test and then beginning to go into production to deliver aircraft to the Air Force. So I was fortunate there to be involved in some of the flight test activities, partly with respect to the flight test program that tested the B-58 to prove all of its characteristics to the Air Force.\n\n Then subsequently, as a part of the—this may be a misnomer to call it flight tests, but essentially the same thing when we roll an airplane off the production line. As it came off the line, you rolled it out on the flight line and you went through a series of flight tests to verify that everything was working right before you delivered it. In both of those, there was, again, a similarity that was kind of an extrapolation of what I'd done in the Air Force. Again, with the exception of flight tests, that began to be a little closer to flight control in NASA. When you were flight testing the airplane, you started out with a flight with a set of objectives for this flight: I'm going to test the flight control system, or this flight might be to test the engines at this time. This one might be to test the performance of the airplane. So you had different characteristics that you were flight testing.\n\n At the same time, you were monitoring the airplane from the ground as well as the on-board crew, just very similar to what we did in flight control, and if something went wrong, you were trying to sort out, is this a deficiency in the airplane design or is this a malfunction in the system or have we messed up in how we planned? You know, is its overall performance just not what we thought it would be? Is the design-deficient? And you tried to finish every flight test. You had a certain repertoire of things that proved that this particular system or tested it and you had planned so many flights to get that done and you were trying to get it done within that number of flights.\n\n Very similar operation to NASA. If you began to have problems, you began to then look and say, well, what else can I do? I don't want to waste this flight. So I see a pretty strong parallel to what we did there. We did not have the same kind of facilities assisting us. We had a much more simplified downlink and in general much less of it was available to us to look at on the ground as was available in NASA.

Kevin M. Rusnak: Any parallels in dealing with flight crew on this?

Robert L. Carlton: I would say an exact parallel. I have a great respect for man's interface and interaction in operations, and yet at the same time, we have a dichotomy here. Man can do things and is a great value and gives you a much better chance of accomplishing a mission. At the same time, man is more prone to mistakes than machine. Somewhere I saw an article on flight safety of airplanes, and I think, if I remember rightly, it was over ninety percent of all aircraft accidents are human error. So you think, well, the last thing you want is a man aboard. But that's not true. When the hardware fails, man can diagnose it and can pick up the pieces and go on with what's left and make do, whereas if you tried to automate the process, when it don't do what it's supposed to do, usually you're dead in the water, you lost that one.\n\n But to more directly answer your question, the flight crew would—very frequently the problem would be human error, you know, something wrong with what they were doing rather than something wrong with the hardware. In flight tests, that was particularly critical. You're trying to qualify the airplane, and it's our human nature, if we've done something wrong, we don't want to 'fess [confess] up. There was always a good bit of give and take between the ground team, who were always suspicious that this problem really was due to human error.\n\n So there's always a good bit of little give and take between the crew and the ground, and sometimes it brought us to animosity between ourselves. Usually it didn't. Usually there was a great deal of respect about what was going on. If you were on the ground and you heard a flight under way and taking place and you heard the pilot report, "This so and so is just not working. I'm doing oscillation," or, "I'm doing this and the other," and you heard the ground flight controller suggest to him, "Well, why don't you cycle this switch, cycle it to the off position and back to the on," amongst the flight control guys there'd be a little snicker, you know. He knew what was wrong with it, but he was being polite to the pilot. He forgot to put the switch where it belongs, dummy.\n\n At the same time, you know, I'm sure they saw and got frustrated with us when they'd have something that was really wrong and we'd refuse to accept it. And especially if they brought it back on the ground and told us what it was and the next flight it's still there. So it's a two-way street.\n\n Now, I went off on a tangent. Did that answer your—

Kevin M. Rusnak: Yes.

Robert L. Carlton: The B-58, I think, was a great learning tool for something else that had a very strong application of flight control, and that is, when you're in a flight test of an airplane, even though it's not as critical as space, there still is a criticality factor there that gets you in the mind-set of you've got to work fast, you've got to solve something quickly, resolve it, and you've got to take action based on it. You don't have time to dally around. Even though it's an airplane and it come back and land, and that does ease the stress, it costs a lot of money to launch one of those beauties and bring it back, and it was not a small thing. It took several weeks to turn around work on it, to get the airplane turned around and back up airborne again. So you tried to do all you could, and it cost the company a whole lot of money if you failed to finish all the objectives of the flight.\n\n From the B-58 flight test, we reached the point where we started to deliver aircraft to the Air Force, and General Dynamics sent a team of people to Bunker Hill and to Carswell Air Force Base [Fort Worth, Texas] with two squadrons of aircraft that they sent out and deployed. I went to Bunker Hill with that team, and there we had a group of young Air Force mechanics. They were learning to maintain the B-58. And a new group of pilots, they had already flown it and gone through training of flight, but they were learning to live with it. So it was a learning experience and quite a bit different from the flight test environment and yet in a lot of ways similar.\n\n But the main thing that happened there was, now, [I was] no longer on line hands-on, I'm more looking over the shoulder, advising and training. That was akin to the later stages, I guess, of flight control. But still, I look back on those days, it still was so similar to what I was doing, but it was less pressure. Air Force were less concerned if a flight came back down. Now, every flight was a training flight, I guess you'd say, in the early days of the deployment, so it was more relaxed and a fun sort of thing from the pressure of trying to get the flight test program over with.\n\n From General Dynamics, I went to Lockheed Marietta—in Marietta, Georgia, and worked on the C-141 [Starlifter], which was in the final stages of design, I worked in the flight control area as a maintainability advisor to the design group, and there I saw a different perspective of aircraft. The shoe come on to the other foot, as the saying goes. On the maintenance side of it, every time we'd run into a problem, we'd cuss the designers, and you'd sit down and try to see what was wrong and what they should have done and how they should have designed it, and you didn't really have a lot of good dots for them when you run into problems. It's kind of like in your automobile, you get in to change a spark plug and you can't get to the plug, you know, you cuss the designer.

Kevin M. Rusnak: Yes.

Robert L. Carlton: That's what we did to the airplanes then. But now the shoe being on the other side of the foot, that was my job on the C-141, was to design in easy maintainability. The Air Force woke up to the fact of the need for that and insisted on it, and when Lockheed won the design for the C-141, well, that was one of the strong points. They had their superior ability to design in easy maintainability.\n\n I wasn't there very long, only six months, but it was in the crucial time that design was gelling and finalizing. So it was a fun thing to do. But it give you—it gave me, at any rate, a new perspective and a new sympathy for the guy on the other side of the fence. It wasn't as easy as I thought it'd be when you're trying to second-guess what are these dumb bunnies going to do on the flight line? How will they mess it up? And as I talked to those guys trying to design, I saw things from their view.\n\n I remember one time we were talking about the hydraulic system, and we had the plumbing go back—and when you have a hydraulic system, you have hydraulic power, liquid under high pressure, and it provides the power to operate an actuator through some controls. Well, that fluid goes into the actuator, but it's got to come back. It comes back under low pressure. So the side that carries the high pressure fluid down there is maybe 3,000 psi and the side coming back might be 50 psi. Well, weight is critical. You don't have to have as big and heavy and stout a line to handle 50 psi as you do 3,000.\n\n But as they were putting this together, I remember them saying, "Now, you've got to use a different-size fitting." They said, "Those dummies will interconnect them wrong if you don't." And so you would have a totally—where it was impossible for you to hook it up.\n\n They told some scare stories about in the past how these guys had crossed them up. One case I recall them talking about the high pressure line didn't fit the other one, so the mechanic decided something was wrong and he went and had himself the hoses rebuilt down in the hydraulic shop so they'd hook up where he thought they went. [Laughter] So you only see it from your own viewpoint.\n\n But it was interesting and fun, and it did give you the perspective to look from both sides and to be a little bit more understanding when it wasn't designed like you thought it should be from an operations viewpoint.\n\n From working at Lockheed, then, I went to the Army Missile Command at Huntsville, Alabama, and worked on the Sergeant missile. There we were involved in the engineering oversight of the design of the Sergeant. It was designed and being built by Sperry Utah, and that was more remote. It no longer was a hands-on and in the middle of the hardware. The design was almost completed at the time I came on board. They were still flight testing it, but that was by the flight test [operations] guys out at White Sands [Missile Range, New Mexico].\n\n So the role of the engineer overseeing the design work was more the changes they wanted to do to the design, to oversee that and make a judgment of whether it was necessary or not, were they doing it the right way or not, and were they doing it in a way that would be most economical to the government. So it was more a project management hat, which I guess I never thought about to this very instant, but I guess that gave you yet a third perspective of the design of a spacecraft. That perspective was what does it cost and the time to get the production schedule going. I guess deep down that sort of played a role in my thinking from that point on, but I didn't appreciate what a role it was.\n\n After there, I came to NASA. When I worked at the Sergeant Project Office, it came to an end, to a close, as the design was completed, and from that point on, it was just production. So they began to power down the project office, and they would move it over into a different group that was just more oriented toward production advice, where the project office was oriented toward design of a new vehicle, spacecraft.\n\n About the time I was beginning to worry about where will I go next and will they have a job for me here in the Army, I got this phone call from NASA. It couldn't have been more in sync with where I was, and it was a guy named Mel [Melvin F.] Brooks. He died last year. He was my first boss. He was the guy that contacted me. He hired me, and I came to work for him.\n\n They were in the early part of the Gemini Program at that time. The Mercury was still fresh in everybody's minds, and most of the Mercury team of guys had come on board and just moved right on into Gemini. They had hired a lot of new people, but those new people had been on board long enough till they considered themselves pretty knowledgeable. I came in from the outside. They were about the third flight, I think, as I remember, somewhere along in that vicinity.\n\n The first Gemini spacecraft were—well, all of the Geminis were manned, but I got on the Agena, which was unmanned, and it didn't come into the program until downstream.\n\n [Telephone interruption]

Let's see. Where was I?

Kevin M. Rusnak: Well, you had just arrived at NASA and were talking about—

Robert L. Carlton: Oh, yes, the difference in the Agena missions.\n\n As I came in, the Gemini was under way, and Gerry [Gerald D.] Griffin was the Agena. He was heading up a little section of Agena guys that worked on the flight control. We sort of had the Agena divided up into two pieces. In fact, if you look at the way we manned the control center, you see that carried forward. You had one group of guys that got us in control and propulsion systems. Then you had another section, and this was usually a section within a branch. There would be a branch for a spacecraft with two sections, one section G&C [guidance and control] with propulsion. The other section handled the instrumentation systems, the electrical systems and so forth, and the environmental control if it had that on it.\n\n Gerry was responsible for the guidance and control, and he was being assigned to the Gemini. They were hurting for people, and they wanted him to go over there, and he was quasi-trained. I don't know how well he was trained at that time. To me, he was very highly trained. He was at NASA. He represented the only NASA I saw at the moment [except me]. But anyway, he sat down, and he was going to give me—I think I mentioned this in the little write-up—Gerry was going to give me this—they told him, said, "Bob's going to come in and work in your place. So you bring him up to speed on the Agena."\n\n So we sat down, and he sat down and talked about the Agena guidance and control system for about thirty minutes or an hour or so, and the phone started ringing. He answered the phone, and he said, "Well, you're on your own here." [Laughter]\n\n I thought, "Oh, goodness." I told him, "Well, tell me where your documents are that describe this bird."\n\n But that was kind of characteristic of what took place. Now, they did have some training classes that came on line sometime after that. We did get some training classes, but by the time we got them, by that time we'd learned the systems. But they helped. When somebody came in and gave you a long detailed description of the system, he saw stuff that you hadn't really appreciated, and also the training people do it so much till they kind of tie it all together, and the engineer tends to get all bogged down. He can't see the tree for the leaves. As a matter of fact, he can't even see the leaves sometimes for the veins in the leaves. Ask him what the forest is, and he don't even know there is a forest there. But anyway, we spent quite some time learning the Agena.\n\n Those early months—I can't remember how long it was from the time I came on board until we flew the first flight. It probably was a year or so. I came in November of '64, as I remember, and I don't remember when the first Agena flight was. You've probably got it in your records. But my recollection is we had about a year to get up to speed.\n\n In that year, we hired some additional people. When I came on board, there was myself and Ed [Edward L.] Dunbar [Jr.] in the Agena area where I was. Ed Dunbar and Bill [William E.] Sturm, I think, came on board at that time. He was a Philco [Philco Corp., later Philco-Ford Corp.] contractor. Ed Dunbar was also a Philco contractor. The way we were organized at that time, the Philco contractors manned, for the most part, the remote sites, as far as the systems monitoring was concerned. You had three people that went out there, a capcom and then you had a Gemini systems man and you had an Agena systems man. The capcom might be a NASA man. As I remember, some of them were at Philco, or sometimes there might be an astronaut out there was a capcom. We had Philco people that worked with us in the control center. A lot of those people had been with them through Mercury, so they were pretty proficient, pretty skilled in what they were doing.\n\n Then we began to hire people. At that time, they were already looking ahead to Apollo. Gemini was a stepping stone to Apollo, and from the very first, from the time I got there, that was understood. So the hiring was preparatory to Apollo, not really just to do Gemini.\n\n We had a situation in retrospect seems kind of ludicrous to me, but at the time it never dawned on me to question it, or anyone else. The manning for Apollo was against the supposition of many, many flights and a high flight rate, and I don't recall the number of flights, but I remember over and over we were told, "Now, you're going to have multiple flights coming back to back to back. You'll be working people too hard. You need three full shifts of people to do the job and maybe a fourth. They can be off line having a vacation."\n\n Now, if you go back and look, we didn't have very many Apollo flights. What happened was, the public began to say, "This costs too much money," and the Congress reflected that. So the Congress began to say, "Hey, shut this thing down. Shut this thing down." So just came the cleaver, we shut down. We had planned to have a high flight rate and power up with a lot of people to sustain that flight rate. Fortunately, we didn't get powered up fully, so that saved our bacon.\n\n But back in the Gemini days, we were looking ahead to that and beginning to try to bring people on board so we could give them training in Gemini that would carry them on in to Apollo. In my section, we brought on board John [A.] Wegener, and another young man named Paul [D.] Nering. I'm having trouble remembering—about the time we finished Gemini, they had meanwhile had hired an entourage of people who were following Apollo, and they moved in with us. We had several Air Force people came in and worked with us. I was just trying to remember if they came in as a part of the Apollo group or we hired them, and I think when I say we hired them, the Air Force would send them in to train with us. So we didn't really hire them.\n\n They had topnotch people, by the way. They must have skimmed the cream of the crop off the Air Force people to send to us. I never saw an Air Force guy that I just didn't really get impressed with. I had Bruce [A.] Stach. He was there during Gemini. Ed [Edwin E.] Marzano, Jerry [Jerome M.] Sisk, Hopkins—I know Hopkins, Carroll [E.] Hopkins, came in as a part of the Apollo. It may be Marzano came in, but I believe that the others came in directly assigned to us as they came in from the Air Force, if I recall right.\n\n But we were powering up at any rate, is the point I'm developing here. During the Gemini we were constantly powering up, and that had several effects on us. One thing was, in the front end of the program we were just so swamped. That few bunch of guys, we were trying to put together our drawings to fly the missions with. That was a very time-consuming thing. Did you ever see any of the mission rules preparation in NASA? Do you know what mission rules is? Have you been through that with any of the other interviewers?

Kevin M. Rusnak: Yes, on a basic level. But why don't you go ahead and explain it for the record.

Robert L. Carlton: Mission rules took a tremendous amount of time, but it was a very powerful tool that NASA had, and I think it was a unique tool. If we'd had it back in the aircraft flight tests, it would have been very, very helpful. It served a variety of purposes. What a mission rule was, was you sat down prior to the mission and you said, "Let's make some rules up of 'what ifs' this happened, what would we do to react to it." "If this failure occurred" was usually the way they were oriented. So you start at the front end. You say, "Now, what's most important about this mission?" We want to get as much as we can out of the mission. A lot of money is required to put it up there, and a lot is at stake, and you don't want to waste any of it. You want to maximize the return you're going to get out of it, get every experiment done you can possibly do, every flight test item done you can possibly do.\n\n In conjunction with that, you have got to protect crew safety above all else. So you're starting in, a mission rule would be that you will never do anything to jeopardize crew safety. I mean, you didn't have to really write a rule for that, but it was there, and it was your starting point. You had a rule that was an offshoot of that, to protect crew safety, you always tried to have redundancy in your systems so that if one system failed, you still could bring them home with the other systems. So you had a mission rule that said you will always maintain yourself in a posture such that no single failure, subsequent single failure, could put the crew at risk.\n\n Now, to illustrate how you would apply that, if you had a reaction control—RCS— system—you know what that is; that controls the attitude of the spacecraft—and it had redundant propellant supplies to it, and sometimes those two separate paths would have redundancy in themselves, and you might have a failure in that second set, a level of redundancy. Well, at that point, maybe you had a plumbing line going from a propellant tank down into the system, and maybe it went through two valves, you had two check valves. When it flowed at this and stopped up, it would still flow on downstream. If one of them stopped up, okay, I'm in a posture where one other failure will kill this system. However, I've got another whole system. So this failure don't get me in a posture where one other failure would kill me.\n\n Or it might be that maybe it would be in a pressurization system that flowed gas to pressurize a propellant tank that in turn pushed the propellant out to the engines. Well, I've got valves in that pressurization system that's pushing gas down on top of the propellants. As I use propellant during a system, the propellant tank, you know, it's a big tank, so you empty it. Pretty soon it gets where it's over half full of gas or two-thirds full of gas. Now, the gas coming into it is regulated, and as you do burns and you use the propellant up, more gas comes in and keeps the pressure.\n\n Well, you reach a point, though, gas expands. So you reach a point about halfway point or maybe a little more, where if you lost your gas, you've got enough expansion ability in there that it would push the rest of the propellant out. So at the front end of the mission, you'd have a mission rule that said, "If I lose this, I come home quick because I've got the other system, and that's all that's got me. I've got one system failure between."\n\n But later on in the mission, as you get more and more volume in that tank, you'd reach the point where, "Aha. This tank will bring me home. Now I've got two systems to bring me home." You can just do anything you want with that. So mission rules you apply to failures to help you take corrective actions that maximizes mission success.\n\n That's basically what they do, but as you sit down and start planning how you'd react to a failure, say we've got three experiments, and one of them takes pictures of the surface, and one takes pictures of the sky, and one of them does something else, and we've got another experiment that conflicts with that. This experiment to take pictures wants you to do attitude controls, and I've got another experiment that wants you to not do it, it wants to be real stable. So I have to decide which one of these is most important.\n\n If I have a failure that's going to make me come home early and I can only do one experiment, here I am, a little poor flight controller on a console, and I don't know which one of these two experiments is most important. Which one do I give priority to? Well, that's above my job level.\n\n The manager that's managing the whole project makes that determination. So he gives me a guideline set in the mission that becomes a mission rule. Here's the priority of all of the things we do, not just experiments but everything. So the mission rule becomes a mechanism for him to tell the operators the priorities of what needs to be done. Then when we little flight operators then begin to look at all these failures and postulate what we'd do, it becomes a mechanism to go back and review mission rules with the managers for them to see how we're going to react to a particular failure. So it gives them insight into how we're operating the mission, and it gives us insight into how they want it operated, and it gives us guidelines as to how to quickly react to an emergency. A very, very effective thing is a mission rule.\n\n Another thing it does, it trains you. A newcomer coming into the system, and he has it explained to him what a mission rule is all about and what you're supposed to do with it, and the first thing I would do with my guys I hired, I'd say, "Okay. You're in charge of the RCS system. You're an RCS expert. Here's our mission rules for this next flight. Go through and tell me all the failures, now, and how we're going to react to them, and I expect you to understand what we're doing in this flight, and these mission rules tell you what we're doing."\n\n Now, he also had the flight plan that outlined every single thing we're going to do. We'd already gone through it and laid out a total flight plan, and everybody had the mission rules available to them, and they worked together with the flight director and with mission management to put together a flight plan that accomplished the most. And then the mission rules allowed you to change that flight plan according to failures that were incurred.\n\n So I'd tell this new guy, "Okay. What if you lose this propellant tank of gas?," you know, what if, what if, what if, what if, and he would come back and propose a mission rule, "Here's our corrective action if we'd had this failure." I could look at what he proposed, you know, and I could critique the heck out of him. And in that critique, I'm teaching him flight control. I'm teaching him what a mission rule is all about. I'm teaching him to orient his thinking toward maximizing the mission's success, but don't ever jeopardize crew safety. So it's a very effective training tool, as well as that communication, and it allows the manager to control what's happening, even when he's not there.\n\n Then last, when you go through mission rule review after review after review and planning and planning and trying to decide what you're doing and in simulations, applying that mission rule, then comes the day of flight and a failure occurs, and I've never seen any statistics, but I'll bet you that probably somewhere between a third and a half of all failures were not failures we'd thought of, that we had no mission rule for them. But having gone through them so many times, you got to where you'd think along the same lines, and not only that, as a team.\n\n This is something else that gradually comes out, and it takes a long time to absorb it and to appreciate it; the flight control team works like a team. When it don't, it's not very effective. It has to work as a team. When all of the guys that work together as a team, they all think alike on what we're trying to do, this mission rules helps mold us into a team as we applied it in simulations and as we then applied it in the flight itself.\n\n Well, the mission rules was something that helped us make that transition in the Agena to becoming knowledgeable about what we're supposed to do. We finished our drawings, we had the mission rules, we worked on the flight plan, we worked on the ground systems where it was being built, they brought in the mission control system Houston during the Agena Program. The early days, they were flying from the Cape and they were working furiously, the ground guys were, to get the MCC [Mission Control Center] ready.\n\n And what did they design in the MCC? It was what the flight controllers wanted. We gave them a set of requirements, said, "This is the way we want the console laid out." Now, when I came on board, they already had the basic concept set in concrete. You would have TV tube systems. You'd have these little event lights. I don't know if you've looked through some of those consoles, but around the perimeter of the TV tube you had a whole bunch of little event light panels. The first sets had thirty-six little event lights, and very quickly we filled those up. Somebody went out and got them where you split each one, and I've been able to have seventy-two. That was great.\n\n Learning to use the TV tubes was something that was a learning experience. In flight test work, you had banks of needles. Each needle represented a particular parameter. Arnie [Arnold D.] Aldrich, he'd been using the needles down at the Cape. They had needles, or little gauges. Well, displaying your system on the TV tube was a gigantic leap forward.\n\n On my system, I had it laid out in a schematic, just like you was looking at your schematic drawing. The fuel tank's on the left, there was a little bubble. Inside the tank was a little measurement that said 50 psi or whatever the pressure was. You'd just look in there, and there it is inside the tank. Coming out of the tank was a line that represented the gas line going to the propellant tank. This was gas that was under high pressure, 3,600 psi, and it went through a valve, and then it went through a pressure regulator that regulated it down to about 200 psi. From there it went into the tank. It kept a blanket pressure on the tank of 200 psi. As you burn propellant and as the propellant emptied the tank, it just kept putting more pressure, and then you used that pressure—you didn't have a fuel pump—you used that pressure to blow it out. Well, you had that schematic on your TV tube.\n\n Now, Arnie—I almost hate to use the name, because, you know, we've all got something downstream we did that's funny. It wasn't funny to us at the time. Arnie, I looked at his TV tube, and first off, they told him, "You've got to take all those gauges off at the console."\n\n "No. I won't do that."\n\n I could empathize with what he was saying. That gauge was instantaneous. It was high data rate, very sensitive. If the pressure was doing this [gestures up and down], that little needle on the gauge is doing this. The TV tube, the data come into the big mission control center computer, and it put it in a database, and then plop, it updated your number. One second later, plop, the number changed. So it was very slow compared to the dynamic response of the needles. Well, Arnie said, "No, I don't want that. I want the needles, make them on all of my instruments."\n\n "No, you can't have it, Arnie."\n\n Well, the first configuration, he had both, and guess what was on his TV tube: pictures of the needles. [Laughter] Well, he could only do about five parameters on his console. So what did he have? He had a very degraded—you know, it didn't even compare to his needles, but it only had four or five parameters. I had 300 and something on my display. And you could get a lot of data on it once you began to appreciate it.\n\n Well, after he got to using it, you know, he made this transition, but it was kind of funny. I cackled at him many a time looking over there at that, and there's that big old TV, or that big old ugly meter. But he got swamped by progress and learned how to use his system.\n\n Well, as we started on in to our first missions, we finally got ready, and our first Agena mission was the Gemini VI spacecraft, Gemini VI mission. It was the first Agena spacecraft. And this little beauty, we launched ahead of them. We got in orbit and we were their target. Then when they'd come up with the Gemini spacecraft, they'd launch behind us, because I remember they used the next rev [revolution], if I remember rightly. We launched, and we went around one rev, and when it got in the right position, coming overhead and zoom, here they went after it, chased us, and did a rendezvous with us and docked with us, etc.\n\n Well, we started into orbit. We launched, and we started toward orbit, not into orbit yet, and we were sitting there watching the data, you know, a bunch of green—first flight for any of us. We'd had a lot of simulations. Bernie [Bernard R.] Brabant was the electrical guy in the back, I was the Systems guy, and Mel Brooks was the Control. He was the guy that sent the commands, and he was sort of the boss of the—we were, two of us, on the front room.\n\n With the Agena, I had all the systems. I guess it was the only spacecraft where we didn't split it up into guidance and control and electrical systems, and he was the command. He took care of the command. The Agena we commanded from the ground. It was unmanned.\n\n Well, as it went up into orbit, what happened was, I'd have to draw you a picture for you to see it, but fundamentally what happened was that when we bought the Agena, we changed the way the propellant was put in the rocket engine.\n\n Now, the way the engine was designed to work was in the inside—are you familiar with a rocket hypergolic? The hypergolic propellants, if two propellants touch each other, they burn. You don't need a sparkplug like you do in a car. So the way this rocket engine worked is you had a big flat plate about this big around, about this thick, and then the engine bell is around that. This is the injector fuel. It had a jillion holes drilled in it.\n\n And then behind it, you had—I guess you'd think of them as feeder tubes drilled in it for oxidizer, a big hole. So the oxidizer would come squirting out. If I just took one of those—there's three holes to each thing. You'd have one oxidizer, and then you'd have the fuel come in at an angle and impinge on this jet oxidizer, and that was repeated over this whole housing. I don't know how many holes was in it, but a bunch, and you had little manifolds behind it drilled through the plate. So these feeder holes penetrate in that manifold. So in the manifold holes behind it, you had this oxidizer with a bunch of oxidizer holes coming out. Here come a jet oxidizer, and then the fuel would impinge on it.\n\n Well, what happened was, the Air Force, when they cranked the engine up, they squirted the oxidizer out first, and then milliseconds behind that come the fuel, and you got a good steady stream of oxidizer going and then the fuel starts impinging on it, and as it does, of course, it starts to burn and create big pressure and then blows out the engine, you know, and you've got combustion and you've got thrust.\n\n Well, for some reason, it was before my shift, but, you know, this was already designed when I came on board, when Agena went into orbit, when the engine cracked, the telemetry went dead, just like that. Now, it took a long time going back and looking at strip charts to figure out why it went dead, and I didn't understand why it went dead till years later.\n\n What happened was, NASA changed it to be what's called a fuel lead. Those two jets of fuel turned on first, then come the oxidizer out. Now, I'm explaining something to you that I'll bet you there's not three people in this universe really understands why that engine blew up. If you take two squirts of the liquid and squirt them here, you know, right against each other, they'll hit and go out, won't they? You can see that. They'll realign and come out. Now, if I start changing the angle on them, it still will squirt out. Finally, you'll reach an angle where there's no squirt back this way. You see what I'm describing there? It's all this way. Okay. I'm not to that angle yet, okay? I'm on an angle about like this. I'm still squirting some this way and some this way. So here comes my fuel. Squirt, splatter, splatter. Now, what's down here?

Kevin M. Rusnak: Oxidizer.

Robert L. Carlton: Oxidizer hole. No oxidizer there yet. So I squirt that hole full of fuel down in the manifold. Now I've got a bunch of fuel down in the manifold, no oxidizer yet. Now, when oxidizer comes rushing down that manifold, what's going to happen? It's going to blow up. It's going to blow up. That's what happened.\n\n At the time, when we went back and looked through it, looked at the data afterwards, I didn't know what I've just explained to you. I knew that we had an explosion in the manifold, but I didn't know how the fuel got there. I thought probably we had a crack in these manifold lines, was what I really thought, but I didn't know why. But I saw the explosion. We had a bunch of transducers, and I could see the explosion hit this transducer. It popped out. Milliseconds later, this transducer popped out. Milliseconds later, this—and I could backtrack it all the way back up, you know, pop, pop, pop, pop, pop. You could see the shockwave go ricocheting up through there, and this is milliseconds.\n\n Now, it didn't take long for the shrapnel to kill the electrical power. Now, it was still milliseconds. You know, if you were looking at it real time, it's like everything just turned off. It took maybe three seconds or four before we lost total telemetry, but so much happened so fast. And what happened, the power started down, and that drew our attention, and we started looking at that. Then telemetry turned off, and we didn't realize what had happened to the engine. So when the engine blew up, in the meanwhile, this propellant tank's being pressurized, and it overpressurized, you know, so the whole thing just disintegrated.\n\n The way I realized why it did like it did that I just described, later, years later, I was out at White Sands Missile Range and I saw how they tested the Agena with this fuel lead, and the way they tested that Agena engine is they pointed it downwards. Now, as they pointed it downwards during the test firing of the engine, you've got gravity here on the Earth. These two fuel jets are pointed downwards. Now, this splatter that goes backwards into that oxidizer hole, it's working against gravity to go uphill. You didn't have any gravity in space. So when we did a test firing pointing it downwards, we did not uncover the fact of what was going to happen. I don't think. I don't know. When I saw that, I just thought, "Well, you know, now I understand how the fuel got in the chamber." I don't know if that's understood anywhere or not. Well, anyway, so we lost our first Agena.

Kevin M. Rusnak: Can you tell me what was going on in the Mission Control Center when this happened?

Robert L. Carlton: Well, the thing lifted off, had a good liftoff, everything was normal, we're going toward orbit. Gene [Eugene F.] Kranz was on the console. He was our flight director. Mel Brooks and myself were sitting down on the console. And we had across the front—I'm sure you've gone through how the control center was laid out. The flight dynamics people are keeping the trajectories and so forth. They were there. The Gemini guys were there waiting. They were going to launch in a few minutes themselves.\n\n When the thing started having problems, of course, we reported it to Flight, and what we told him at the time was that we saw the power loss coming off, and we reported that, and then the instrumentation. This is just both come within a few seconds of each other.\n\n And incidentally, Bernie Brabant was the electrical guy in the back room, and this points out the interaction between—I talked about team work earlier. There's only so much your human brain can assemble. If you sit down and look at a bunch of systems, you only see one at a time. If one of them does a little something funny, you begin to watch it to the exclusion of everything else, and rightly so, because that's probably where your problem is. And the way we work it is, you've got a guy in the front room that's sort of looking at everything, you've got a guy in the back room that's looking at just one subsystem and another guy looking at another subsystem and another guy looking at another subsystem. So you've got two sets of eyeballs watching this thing. You've got two people.\n\n The guy in the back room ought to see it first because he's only looking at a subset of what's going on. Now, if I carry this thought another step further, let's go on up to the spacecraft. That poor old astronaut, he's got all the systems, and he's flying it, and he's got two or three systems he's got to learn. He's got to understand the booster while he's sitting on it and riding it. When he gets off the booster, he may swap the spacecraft. He's got to know the Agena and his Gemini. So the further up the chain you go, the less the guy's got the ability to know everything, and the further down the chain you go, the more detail he can bring to bear on it.\n\n Well, Bernie was looking at the electrical system, and when the electrical power went, the first thing we saw in telemetry was the electrical power. We hadn't realized the engine blew up yet. Bernie said, "Bob, we're losing electrical power," and I think it was about three data frames. We went back later and played it. It's been so long I've forgotten, but I think that after the first data frame that hit his console, he was telling me about it, and then the instrumentation very quickly thereafter, and then the thing went blank.\n\n After it went blank, we had something called a SMEK [Summary Message Enable Keyboard]. Has anybody talked about a SMEK? Well, a SMEK was, you pushed a button, and down in the ground, the bowels of the ground system, it snapped you a picture of what you were looking at at that instant. So we would do a playback and SMEK it. In lieu of having strip charts and all of that brought to the front room, we could just make four or five SMEKs, and we'd get a pretty good time history of what was happening.\n\n So the bird went blank and had ground systems down there go back and replay that data for me, and I SMEKed the heck out of it, you know, and they sent it up to me in a P [pneumatic] tube, and I had it spread out on the floor there, and that's when I realized what had happened to the engine. I told Kranz, I said, "Forget this bird."\n\n In the meanwhile, now, Kranz, he's on the loop, and we had the worldwide loop up. We had guys on the ships covering the ground sites, these ground teams, deployed. And Kranz, he's got the ephemeris of what it should have been had it gone on into orbit without telemetry. He's making the assumption it made it to orbit and we just lost telemetry. So he's got them all powered up, all the way around the world, waiting for this thing to come overhead, you know. What's going to happen is a bunch of shrapnel fell in the Atlantic. But Kranz would not give up. [Laughter]\n\n And after I got all the SMEKs spread out on the floor and I saw what happened to the engine, there was enough there to see the thing blew up, and didn't know why yet. In fact, it was a long time before we figured out why. Well, he had them powered up and looking for it. I told Mel, I said, "Mel, what's he smoking? This bird is gone." But Kranz wouldn't give up. It wasn't until a full lap had elapsed and no sign of it until he finally surrendered, and probably Mel took some of the SMEKs up there and showed him. I don't remember. It's been too long.\n\n Anyway, on that flight, of course, we were dead in the water. Then the Gemini guys went on, they scrubbed for the moment. They had no target to go after. They ended up, if I remember rightly, they had two Geminis rendezvous with each other.\n\n Then the next flight, well, we were in a big panic to try to find out what was wrong with the Agena, and it took a while to happen. So we missed the next flight. The next flight, McDonnell Douglas put together a little target vehicle [ATDA, Augmented Target Docking Adapter]. What they did is took scrap Gemini parts and just threw it together right quick to make something that would get up there and make a target and put a docking adapter on it. Then they sent it up into orbit, and that was to be their target in lieu of an Agena.\n\n The next flight—let's see, Gemini VI and VII rendezvoused together, two spacecrafts, so I guess it would have been—no, Gemini VIII. Yes, VIII would be the next one we flew. Well, somewhere, I forget which Gemini it was, whether it was VII or VI, but somewhere in there, in that interim period of time, they had this docking adapter for target. Tom [Thomas P.] Stafford was the crew member that flew up. He called it "angry alligator." The shroud on it didn't come apart. Somehow it stuck. More than that, when it got up, when they turned on its attitude control system, it was like a dog shaking, and it just hosed out its propellant. I mean, the tank was just emptying.\n\n I remember I wasn't working on it, but Mel come running in, and said, "Hey, Bob, come in here and look at this and see if you can figure out what's wrong with this thing." They had shut it off. They hadn't rendezvoused with it yet, and they'd shut it off before it hosed out all its propellant, still had some left to try to hold attitude, but as it held attitude, it just had the shakes. I never did see the report of what explained that, whether it was something wrong in the attitude control. It appeared to me at the time that they had used a thruster that was oversized. They'd used one of the Gemini thrusters, and the Gemini weighed a whole lot more than this thing did.\n\n If you've got a mass in orbit and you want to attitude-control it, if you hit it with a thruster to cause some momentum, the amount of speed of that momentum that you give it, that rotation you give it, the bigger the thruster, the faster it goes. Or conversely, the lighter the vehicle, the faster it goes; the heavier, the slower it goes. So if you have a thruster that's too big, even though you give it an absolute minimum impulse, it's still going to go real fast. Well, the spacecraft has a dead band. When it reaches a certain limit, it fires the other thruster. So I think what it was doing—I never did see a report, but I think it was just doing this, you know, fire thruster, bang, fire thruster, bang. And the thrusters are firing so quick till, you know, you're just hosing down the propellant. I think that's what it was. But at any rate, for whatever reason, it was all academic, because the alligator jaws wouldn't turn loose, so we couldn't dock with it anyway.\n\n After that, we flew the Agena mission. The Agena finally come to bat. On that mission, we put the Agena into orbit, it made orbit good, we now had an ox [oxidizer] lead, went back to ox lead on our fuel, on our engine. We made orbit. The Agena was looking good. Neil [A.] Armstrong and Dave [David R.] Scott were in the Gemini, and they launched and did their rendezvous maneuvers, and they came up to it and they docked to it.\n\n After they docked, now, in those days, you came in sight—we didn't have continuous contact like we do today. That was sometime later that we got continuous contact. But to have continuous contact, you had to have a way to flow the data from every site back to Houston, and in those days you had a remote site, and we couldn't flow their data live continuously to us. What we had to do was, we could send it like a teletype message. So we had the crews on board. That's why we had the remote site crews all around. So when you launched, the thing come overhead, that's a remote site, and the crew that was on the ground could see it for the duration of their pass.\n\n In low Earth-orbit missions, a pass probably is about eight minutes on average, no longer than about twelve. Now, the higher you get it, of course, the further it has to travel in its arc and the longer it takes it to do it, and the longer you can see it. But when you're down in the altitudes we were in of over 100 miles, between 100 and 250 miles, well, you were in the matter of passes being seven to twelve minutes, depending on the altitude you were at.\n\n So when we launched, we could see it continuously as long as it was in sight of the Cape telemetry, and then it went over Bermuda. Seemed like we had Bermuda data, and we had continuous data to Texas, I believe, and maybe California. I've forgotten whether we had continuous from California or not. But the rest of the world was just little windows as it flew overhead.\n\n Well, Armstrong and Scott caught it, they docked to it, and as they came over one of the ground sites, seems like it was over in Australia, we had a team there and it was doing good, and they were getting ready to do something, and then they came over one of the ships. We had a ship out there. When it came over the ship, it was wound up. Now, what had happened was they had a thruster malfunction in the Gemini, it turned on. Turned out it was an intermittent on; it wasn't solid, continuous, all the time. Whenever it would start winding up, the crew thought it was the Agena. So when they finally got it [stabilized] enough to where they could, they undocked it from the Agena.\n\n Now, they had an emergency I guess you'd call it a return system of RCS for attitude control, and they had the regular RCS for on-orbit. This emergency—this return, or reentry, module was much limited in volume and in total delta availability. Well, what happened was, as this thing started spinning up, it was in the main system of the Gemini, and the thing started winding up on them. Well, Neil Armstrong was fighting it. He was opposing that failed thruster, and Dave Scott was over there pulling and pushing circuit breakers. By this time, I guess, they knew a thruster had failed on. If you could talk to them, by the way, to see their perspective, that would really be something to add to—have you been able to interview Armstrong and Scott?

Kevin M. Rusnak: Not yet. They're both in the works to interview, but we haven't talked to them yet.

Robert L. Carlton: Well, anyway, if you looked at what they were doing as it come overhead, over that ship, the thruster was on and off and intermittently firing. On the ground, Gerry Griffin was the systems guy on the ground on that ship. Now, I've forgotten who the capcom was. Well, the ground systems—he had all of the spacecraft systems, and furthermore, those remote sites didn't have much—you know, trying to interpret the data is tough. You had a limited amount of display. I had needles, a few needles fundamentally. I'm not familiar with everything they had on there, but it was very rudimentary kind of a system, and it was just impossible on the ground to tell what was going on.\n\n When one thruster fails on, the system, on-board system, it causes the spacecraft to rotate and the on-board system automatically fires other thrusters to oppose it. So everything's on. Which one of them's failing and which one of them—you know.\n\n So anyway, this thing went on, and all we could hear was the voice. They were cutting a teletype message every now and then, but it was slow and after the fact and minutes before it got to you. Fundamentally, it was just voice. When they come over that ship, we sat there just paralyzed with fear. You could hear them, you know. It was winding up, winding up, winding up, and they were at the point where the crew was on the verge of unconsciousness, according to the aeromeds [aeromedical doctors] later. Of course, I didn't know that listening to it. I just knew it was winding up and getting faster.\n\n What I remember the most vividly was Griffin talk about the RCS propellant, it's so and so, so and so, so and so, so and so. He was reading the main, I guess, instead of—and then he said they're on the reentry, they've activated the reentry. So then they went out of sight.\n\n Now, that was a thriller. Scared. I mean, it sounded like we lost them. I sat there listening to that, and I thought, "Ah-oh, oh, man, Gerry, tell them which thruster, tell them which thruster, tell them which thruster," but he just didn't have the data to do it. As I sat and listened to that, I said to myself, I said, "I guarantee you I will have my console arranged in such a way that I can tell a failed thruster. If this ever happens to us again, I guarantee you I'll be able to tell them which thruster failed, and I mean now, right now."\n\n I was paranoid about it from that point on, but I found out—you know, resolving is one thing, doing something else, it's almost impossible to do. What makes it so difficult to do is two factors. One is that the system turns on other thrusters to compensate for the one that's failed. That mass—you know, the effect to you when you look at telemetry coming out of the bird, the ground systems can't handle all of the telemetry at the high data rate it comes out of the bird. So what they do is, at the remote site, they will slow the sample rate down and send it to you like, maybe—well, something comes out of the bird at a thousand times a second, you might see it one time a second. Well, in that one second, it may have changed configuration three times. So you can't—and even so, your eye, you know, you can't think much faster than about two times a second. So it's hard to do.\n\n The effect was, when it come in sight, you're going to see every light turned on. That's fundamentally what the net effect of it is. We worried with it and worried with it, trying to do something in the ground control system to give us the ability to detect a thruster.\n\n One thing we did was arrange the thrusters. We had these event light panels. So say you're looking up the tailpipe of the bird, and there's a thruster here and a thruster here and a thruster here and a thruster here. Now, if you wanted the bird to turn this way, you'd fire this thruster and this thruster so they'd be in a balanced pair and it would cause it to rotate. Or if you wanted it to go the other way, you'd fire this thruster and this thruster and cause it to rotate.\n\n Now, suppose this thruster fired, inadvertently, turned on. Well, whether the on-board crew grabs a hand controller to correct it or whether the automatic system does it, how does it correct for this torque? It will fire this thruster and this thruster. So I went to great troubles to align these event lights up so that these two thrusters were right opposite each other. I visualized here's the spacecraft and here's the thruster. Now, if these two thrusters are firing against each other, that should not be. I know one of them has failed on. So how do I know which one has failed on? The one that's right has got a mate to him down here, okay? So when I see two fire, I know I've got a failed thruster, and my next thing is to see which one it is. Got it, right? No?

Kevin M. Rusnak: No.

Robert L. Carlton: What killed me was the data rate. I thought when I put that on the console and programmed it to do that, I thought, "Man, I've got it now. I can find the thruster every time. Instantly I'll tell them which one it is." So we simulated with it, and oh, it worked beautifully. I was just tickled pink, you know, "We have got this whipped."\n\n And the first flight we flew, and we got up there and the astronauts started their little maneuvers, the whole board turned on. Now, what happened was these thrusters are only on for a millisecond, but the ground site with this data rate mess, it turned the bit on for two seconds. So every time a thruster fired, it turned this bit on, so my light turned on for two seconds and stayed on for a full two seconds. Well, I didn't appreciate that they were doing that to me on the remote site till that first flight come over. So that's what you run into. Well, we finally got that squared away, though.\n\n Then, the other thing, on the Apollo 11—I'll jump from the Gemini to 11, but I'll show the application of what took place. On the Apollo, not only did you have these thrusters that were fine-tuning your attitude, but the big main engine, the landing engine, could gimbal. So if this thruster fails, it's going to cause the spacecraft to rotate. Now, the other thrusters will turn on, granted, but the gimbal of the engine, when it sees this rotation, that big engine is also controlling the attitude, so—whup—it'll gimbal the engines. It'll just totally overpower those thrusters, and it'll hold up a dead band—like if this is the dead band, the engine will hold it there and the thrusters won't react till it gets out here.\n\n So the way we discovered this, I hadn't even thought about the dadgummed engine. I was looking at these event lights and also the RCS usage. You know, these different thrusters on two different RCS systems, I can look at which one is using the most gas and kind of deduce that back into which thruster has failed. I know it's in this system because it's hosing out gas at a different rate.\n\n The sim [simulation] guys, they threw us a failed thruster problem. On the sim day that they gave us the failed thruster problem, by then I had it fixed. I had the event lights. But it just so happened that the day they threw this at us on the sim day, for some reason the ground systems was messed up and my event lights were not working that day. Well, this was the day to really work this over, and they threw me failure after failure. Oh, that was a nightmare sim day. They just kept failing thrusters and failing thrusters and failing thrusters.\n\n Kranz said, "Carlton, what's wrong with you?" [Laughter]\n\n There was nothing I could do. I said, "As long as they fail the thruster, we're going to crash the mission. I just don't have any way to detect it." Well, they had their script already made up, and they cut some of them out.\n\n But about that time, one of the guys in the back room—I don't remember which one of them it was—said, "Hey, Bob, look at what the gimbal's been doing." And he got to looking, and what would happen is, this thruster turns on—whomp—you know, the spacecraft starts to rotate, the big gimbal, and that gimbal would go cattywampus off way to one side, torquing against the thruster, and then it would land. The instant you saw it, you knew, "Well, why didn't I think of that before?"\n\n Well, that's the way a lot of the flight control things was. You know, the failure, I said a minute ago, about half the failures, you never thought about them. Half the solutions you didn't think about ahead of time. You evolved to them after you thought about it.\n\n Once they realized that the gimbal was going to do that, in the middle of that sim, those guys back there were working fast and furious, and before the sim was over, they had this little skinny sheet out there, you know, we had it, and then we were telling sims, "Throw them at us. Let's practice this a while," you know. We got to where we could tell which thruster it was just instantly just from the gimbal, and never even thought about that before.\n\n Well, come Apollo 11 and we're landing on the Moon, we're two thirds of the way down the trajectory landing on the Moon, and my event light popped up, a failed thruster. About the time I saw that, I saw him take over the hand controller, and then all the thrusters started blinking. I'm sitting there using my logic, but I went to the gimbal and looked down there, and the gimbal was just as steady as a rock. Now, what did that tell me? It says there's no thruster failed. If there's a thruster failed, that gimbal would be off in the left corner. There was no thruster failed. It just, that quick [snaps fingers], had it.\n\n If you go back and listen to those tapes, you will not hear one word about that thruster. If you listen real carefully, you'll hear me tell Bill Sturm—he was the guy in the back, he hadn't even noticed it yet, I said, "Bill, when you notice the thruster, it’s all right, that's the instrumentation," and we didn't mention it on the loop. We let them land. They were busy as could be and thought that we don't want to muck up—if we start talking about that, that's the wrong thing to do. They had a handful trying to land, and I don't want them worried about the thruster. So we said to ourselves, if they have another failure or if they notice that thruster—they had a little malfunction thing on board that would tell them a failed thruster. Well, if they notice that thruster, I'll tell them if I have to, but till then, let's just let it ride.\n\n Then after we landed, we went back and talked about it and were prepared for it and changed some of the procedures just in case. So that was application of that to the Apollo.\n\n Now I go back and every time I see the sim guys, I tell them, "Boys, you saved my life. You saved my life." [Laughter] And they did a lot of other things like that, the training.\n\n That points out something else I think is an experience learned, a value learned. In every program, at the front end of the program, all of these designers and the contractors come forth telling us, "Oh, we're going to have an automated, on-board automated troubleshooting system, and it will automatically do the right thing. It'll detect any failure and turn on the valves, turn off the valves, reconfigure the system." And I just think to myself, "Well, you're smarter than I was." I don't care how hard you think, you just can't think of stuff. Then you compound it by each system is different from what you've been exposed to. Every one is unique, even though it's similar system to what used to be, the differences make it act different than the designs you saw earlier.\n\n And who are the guys doing the programming? A bunch of young engineers they hired out of college and a few old heads there, and none of them ever had any exposure to facing the problems in the face. They're like I was back on the -141. You're designing to performance, and you just can't do it. Now, every system, and I've watched a lot of systems evolve through the years, the B-58, we told the Air Force—we've got this automated ground troubleshooting equipment, and then when it come along the next generation of birds, we told the Air Force the same thing, and this same thing, and this same thing.\n\n Every generation gets a little better. The modern-day malfunction diagnostic systems are tremendous, but they gained a lot of experience and applied a lot of experience, but I still say, you show me a guy that just going to blithely assure you he’s going to, you know, you'll get this bird, and he'll be designed it to have a total automated diagnostic system on it, and most especially if he's going to have a totally automated control system on it that does everything automatically, the real weakness of that is you can't foresee everything that's going to happen, and you can't foresee the system will work funnier than you—there'll be something different to what you had analyzed, something you haven't thought about, and get up and bite you every time.\n\n Well, back to Gemini, Gemini VIII, when Armstrong came over the next site, the bird was steady and rolling along, and Dave Scott had found out which one of the thrusters was on, and he pulled a circuit breaker on it, disabled it, and they'd hosed out most of the gas in both systems, both the normal system and the reentry system. I don't remember how much was left, but it had got down low.\n\n We had a mission rule that said you come home if you break the integrity of the backup system. So John [D.] Hodge, I believe, was the one that was on the console then. I can't remember now, but it seems like it was Hodge that made that decision. I might have been Kranz. But anyway, they said, "Come home," and Armstrong and Scott argued with them, "Oh, come on, we know what's wrong. We've got it fixed. We've got enough gas here to go for a while." They wanted to keep going, but they brought them in.\n\n After that, the rest of the Agenas worked good, and I guess it would just be a story of more and more of the same.\n\n There was one other mission that had kind of a uniqueness to it. It seemed like it was the one that Conrad flew. It might have been the next to the last one of the Agenas that flew or it might have been the last one. I've forgotten which flight it was. They did a high altitude—there's two more things of interest in the Agena Program.

Kevin M. Rusnak: Well, if we can take a quick break to change out the tape before you get into that.

Robert L. Carlton: Okay.

There are two more things with the Agena that's worth talking about a moment. One is working with a tether. We tethered the Agena to the Gemini. I forget how long that tether is long. The other thing we did was we shot them up into a high orbit, used the Agena to—oh, there's three things I need to talk about. Let me go back to a totally different one.\n\n I believe it was the first Agena we flew, the Agena, after they got through with it and left it, this probably was the one that Armstrong was docked to and undocked from.

Kevin M. Rusnak: Yes.

Robert L. Carlton: Yes. We did a bunch of burns with the thing. The first burn we did, we pointed this thing and we did a burn, and the delta-V ought to took it right there. It didn't take it there; it took it over here. It was like it was pointed wrong. I couldn't figure out what in the dickens. So we got to calculating it out, and it skewed off the thrust—the equivalent thrust—now, the Agena had a big engine, and the engine, it had a lot of thrust. As I remember, it was 10,000 pounds of thrust, and the Agena was a pretty light vehicle, so it took off like gangbusters when you did a burn. The Agena engine bell was kind of sloppy to move and deliberately so. It was designed to burn with a Gemini hooked to it. That docking interaction there was a pretty weak link. So you didn't want it whipping that Gemini out there. You wanted it to be kind of slow to move.\n\n Well, what happened was the Agena had a little bit of a weight imbalance. In other words, the weight, it wasn't adjusted so that the center of gravity [CG] was right along this thrust line. So when the engine started to burn, and it may have been the engine that's cocked off a little bit to start with. It drifted off just a little bit. Anyway, the thrust vector of the engine was misaligned from where the CG really was instead of down through the center line. So when you first fired it off, the thrust vector is going outside of the CG, and that causes a turn. It had an attitude control system on it, but it was little ten-pound thrusters.\n\n So this big thrust vector of the engine overpowered the attitude control, and there's no telling which way it would turn. Well, Mel Brooks was programming the burns, and he said, "Well, the last time, it just went off—" Now, this is before we understood what the problem was. All we did was we did a burn here and it pointed there, and we said, "That's all right. We'll allow for it. The next burn, we'll play like it's going to be pointed there, and we'll aim it that way." Well, we done that, and it went some—and I thought, "Mel, you're going to enter this thing. We never will live this down if you reenter this thing," but we were shooting out of plane, but if it had been a retrograde, we could have brought it in on one of those burns. Anyway, we didn't know till later. It took us a while to figure out what the problem was. We didn't know it when we was doing all these burns in orbit, but we learned about the Agena.\n\n The other thing was the tether. We got up there, and they're going to do this experiment, gravity gradient. Are you familiar with what gravity gradient is? Okay. Well, we're going to hook the Agena on this big long tether, and we'll be going in orbit gravity gradient with one mass down here and one mass up here and hooked together with this tether. Well, the problem was, that tether got to giving it this, you know. They fought that thing and fought that thing.\n\n Finally, somehow or other, it damped out. I don't know what caused it to damp out, but it was a thrill when we did that tethering operation. You know, normally, as a flight controller on the ground, you know what you're doing, you know if something goes wrong how to fix it, and you know what you want to do next. Now guys, when you're hooked up to this tether, and it seemed like it was Conrad, he starts talking about it's doing this, you know, and there's big waves. Brooks asked me, said, "Bob, what should we do here?" [Laughter] I've never been so helpless in my life. I've got no idea what to do with that thing. I had no idea. All I knew to do was unhook from it. That was all I could tell him. But they finally worked it out, and finally I think it just damped itself out. I don't know that, if Conrad and the crew up there got to wiggling and took the dampening out of it or how they got it out.\n\n But I'll tell you one thing, as a principle of flight control, when you don't know what you're doing, you're in a world of hurt. That's just all there is to it. [Laughter] If you don't know what's wrong, you don't know how to fix it. It's a helpless feeling. That's just all there is to it. It's a helpless feeling. You always went into every flight knowing there were a few little things that you felt uneasy about, "I just hope this don't happen." But you did have a little inkling. I don't think there was anything we ever went into, we looked, always tried our best to know everything about the system and knew a little bit about it. Sure as anything, if it fails where you don't know what to do about it, there's just no more helpless feeling in the world.\n\n That bring to mind something else. I think I mentioned this in one of the papers I was writing, the notes to you. You can also rest assured, when you're on the console and in a mission and things are going fast and furious, you're not going to find any help. You can pick up the phone and call to anybody you want, and the guy, when he realizes—it's kind of like me with that wavering thing there, you know. When you realize what's at stake or somebody's life might be at stake, boy, that makes people docile and quiet.\n\n In other words, the point I'm making is, the poor flight controllers on the console, especially if it's a time-critical thing where it's just seconds or minutes, it's on their own shoulders. I mean, it's either going to get done by those guys or it will not be done at all. Don't think that there's any outside help. There's not. Now, if you've got a situation that's got many hours or days, then you can go get outside consultation. But you can't do it on one of these situations like during a launch with the Shuttle in ascent. I'll guarantee you if something goes wrong with a Shuttle in ascent, that it's the team of flight controllers on the ground is the only people that are going to have any ability to—it's on their shoulders. If they can fix it, it'll get fixed. If they can't, that's all of it.\n\n But the other thing that'll happen is, invariably, when all the smoke settles and the mission's over, and then you understand what happened, everybody in the world will come in and tell you what a dummy you were. "Why, anybody can see that," you know. Now that you can see it, anybody can see that. We used to have a saying, "Hindsight, that's pretty good stuff."\n\n I can't think of anything else in the Gemini Program that is worth calling to your attention. I'm sure you've documented the whole program from end to end. In all of the programs we always had a pot full of experiments they were doing, and they run together in my mind, and most of them I don't even remember. They always had a bunch of them, and in any particular flight we looked at them in great detail trying to maximize the performance out of them, but after the mission was over, you had another pot full of them. It changed every mission, and I don't remember one from another. I have no idea, other than we took pictures on it. They took a lot of pictures.\n\n That tether was an experiment. I don't recall we ever used it again. Now, I think in recent years they have done a similar experiment, maybe by the Italians, and they run a wire down, and I think the purpose of that experiment was to see if it generated an electric current from cutting through the fields of force of the magnetic fields. Have y'all run into that? Did it get any electric force out of that?

Tim Farrell: I think it got so much, it burned out the wire.

Kevin M. Rusnak: I know they had a lot of problems with deployment on that. I think they've tried it twice now.

Robert L. Carlton: I don't know. I've lost contact. When I saw them doing that, I thought, you know, "This is similar to what we did on that tether. I wondering if they're using the same principle of a gravity gradient to hold the thing stretched tight," how did they keep it tight. Well, anyway, it was interesting to me. Other than that, I don't know of any instance where we ever made use of it again, but it sounded like a pretty good idea. It also sounded like, if you wound them up, they would pull them out tight on the tether and it would be a good way of station-keeping two vehicles and not have to worry about them recontacting each other and not have to hose out the RCS gas to hold a station-keep. But as far as I know, we never made any more use of it.\n\n I can't think of anything else in the Agena. Now, go back through your notes, and the next time, if there's something that you think I failed to talk about, we should have talked about—

Kevin M. Rusnak: All right. Yes, I don't want to keep you any more today, since I know you've got some things to do.

Robert L. Carlton: Okay.