Anatomy of a Test Flight

“You want me to fly a what?!” It’s my editor calling. She has a story idea. I stall her while I quickly Google the airplane that she mentioned. After all, I’m the “expert.” That’s how most of our flight tests begin….sort of. This space usually features a pilot report in which I chronicle the experience of flying some interesting new airplane. In this article, we’ll take a tour behind the scenes, so to speak, to review how we perform the typical flight evaluations that you read about in these pages.

It’s a unique privilege to be able to test airplanes for Skies magazine, but also a considerable challenge. The airplanes are often unfamiliar, brand new, and represent the best design solutions from a talented team of specialist engineers. My job is to criticize. 

I typically get one hour to execute an evaluation and walk away with a clear idea of an aircraft’s merits and shortcomings. To manage this challenge, I rely heavily upon the skill set from my “day job” as an experimental test pilot. 

How do you quickly take the measure of an airplane in one flight—without bending any aluminum? My approach is to stick closely to the fundamentals of test flying.

The first step is homework. It is imperative to understand the airplane in its context. Why was it built? What market does it serve? One needs to understand the aircraft’s specific purpose. To illustrate the problem, answer this question: “Which is a better airplane, the Boeing 747 or the F-18?” It is indeed a silly question. “Better airplane” depends upon whether one is trying to carry three hundred close friends to Bermuda or whether there is an armed MiG on your tail.

In short, it’s about the mission. But mission can be more subtle than it appears. Take, for example, the recently updated Cessna Sovereign business jet, which we featured in a recent report. There are at least a half dozen competitive midsize jets, some with the same cabin size, but older technology engines. Others have more wing sweep, and trade field performance for cruise speed. Business jets in particular, by virtue of their clientele, seem to be a hugely diverse ecosystem of distinctive species, each hyper-optimized for a very particular market segment. You can’t evaluate the airplane until you see it through the eyes of a potential customer. 

The test flight is about getting at the facts. I have learned a healthy respect for marketing folks. They work hard to ensure that I have the facts about their product. A careful selection of facts, if you know what I mean. Positive, cheerful, happy facts. That’s their job. If an airplane has shortcomings—and they all do—it’s up to me to find them. 

The aircraft’s flight manual is my secret weapon. I run through a series of performance scenarios. The numbers don’t lie. An airplane may have six seats and a 1,200 nautical mile published range, but that doesn’t mean that you can fill the tanks and fill the seats. This isn’t necessarily a shortcoming, but rather an indication of the design’s range-payload flexibility; a clue to the airplane’s personality. All airplanes conform to the technical state of the art with minor variations, and engineers strive to make decimal improvements. It’s wise to understand an airplane’s inherent design choices before venturing criticism. 

In preparing for the flight, a strict dictum of flight testing is invoked: “Plan the flight, and fly the plan.” Experience has taught test pilots that their business doesn’t lend itself to improvisation. Every test flight is carefully scripted, and deviations from the plan are rarely tolerated. 

Where handling qualities are concerned, time constraints demand a quick look at a few key maneuvers; tasks that will highlight the capabilities or weaknesses of the machine. We brief it carefully, then fly it exactly as briefed. For example, during our recent feature on the Viking Series 400 Twin Otter, I remarked about its powerful and effective flaps, but wondered whether those flaps might produce enough drag to make a single-engine overshoot a bit dicey. So, we tried it. Of course, actually performing the maneuver at low altitude is akin to diving into the deep end of a pool to determine whether you can swim. Even if the demonstration pilots were willing, it’s wise to ease into such maneuvers in safe steps. Test pilots call this technique “incremental build-up.” In that case, we simulated the maneuver at a safe altitude, which served to demonstrate the Twin Otter’s fine handling qualities, but without the risky proximity to the ground. 

I think they told me sometime during test pilot training to avoid touching buttons that I don’t understand. Red buttons in particular were to be avoided. In this spirit, it’s sometimes wise to defer to the demonstration pilot whenever something about the machine seems unfamiliar or particularly technique-intensive. That’s why they call them “demo pilots.” A good example is short field landings, which typically require fairly high proficiency, and if mishandled can shred rubber or bend aluminum. Besides, you notice more and take better notes when your hands aren’t full of airplane. 

One of the biggest differences between engineering test flying and magazine test flying is instrumentation. Flight test aircraft are typically brimming with sensors to record all the required flight parameters. Analyzing the recorded data post-flight allows the test pilot to draw colourful graphs. Not so the aircraft we feature for Skies’ pilot reports. The “instrumentation” consists of a kneeboard and pencil. Over the years, I have evolved a sort of aeronautical shorthand to quickly capture every aspect of a flight. It’s a shame I can’t read it!

Technology can certainly help. I wire a digital pocket voice recorder into the intercom system, and typically install a GoPro camera in the cockpit, controlled from an iPad strapped to my knee. These toys are great, when they work! Suffice to say that my mechanical pencil has yet to fail me. 

The result is a very busy cockpit: there is video running on the iPad, notes to be taken with one hand while flying the airplane with the other, discussions with the demo pilot while keeping a look-out, and—of course—the ever-present “sticking to the plan.” It’s a blessing when a photographer is dedicated to the flight; otherwise, that task is yours, too.

Speaking of photography, flight reviews for magazine articles have something else in common with test flying: the necessity for careful risk management. The missions often involve air-to-air formation photography; a tricky undertaking which necessitates that two airplanes share a small chunk of sky. I have learned a healthy dose of respect for aviation photographers, for whom the enthusiasm to get an epic picture sometimes conflicts with my sense of self-preservation. I’m reminded of once receiving a spirited call on the radio to “tighten it up,” only to remember that the demand originated from a person who was looking at me through a fish-eye lens! 

There is no guarantee that things will be easier if you leave the photographer on the ground. In another instance, we returned from a test flight to perform a few touch-and-go landings, only to discover the enterprising photographer standing on top of a Jeep beside the runway. When we didn’t respond adequately to his animated gestures, the resourceful photographer began to transmit his directions on the tower frequency! I never got a chance to thank the air traffic controller for his patience.

Ultimately, the unscientific objective of any flight test is to arrive at an opinion of how well the aircraft serves its target market. There’s no way to quantify it. Arriving at a “calibrated opinion” is more art than science. 

Back on the ground after the flight, I make a point to share my impressions with the company’s representatives; both the good and no-so-good features, as I saw them. Not every evaluation is glowing. My editor reminds me that I can write whatever I want—so long as I’m right. In my experience, if I describe the aircraft fairly in terms of its strengths and shortcomings, the company representatives just nod agreeably.

Engineering design is the art of compromise.

That’s the story.