A key benefit of the TEAMS solution is that it helps Field Service Organizations improve the Quality of Service (QoS) while lowering the Cost of Service (CoS).

Without TEAMS, organizations rely on the intelligence of the individual Field Service Engineers (FSEs) to solve complex troubleshooting problems. However, not all the FSEs are equally smart, leading to inconsistent performance that no amount of training can overcome.

The consistent QoS delivered by the TEAMS solution is therefore a key motivation for adopting the TEAMS solution. But, how do you measure and demonstrate a consistent performance when there are so many variables that affect the field service process? More to the point, while variability is to be expected, how much variability is cause for concern?

Fortunately, quality measurement is a mature science, and we need to look no further than “Control Charts for attributes“, originally conceptualized in 1924 by Walter Shewart and extended by Edward Demming.

The underlying theory is very simple.

Supposing you want a consistent success rate of S%, where success implies the Guided Troubleshooting Solution enabled the FSE to identify the root-cause of the failure and apply the appropriate corrective action. How would you periodically verify that you are still achieving S% success rate?

If periodically you sampled N cases, the standard deviation, s, associated with the estimate of S from N measurements is inversely proportional to the sqrt(N). This makes intuitive sense; the more data you have the more precise your estimates of S will be.

Assuming the cases you sampled are independent of each other, the expected number of successes from N trials will be between (S – 3s) and (S + 3s) with 99% confidence. This is basic property of normal distribution, and we will assume that it is applicable to S.

So, what does this all mean? Let’s plugin some values to make sense.

Let’s assume your goal is to have a success rate S = 80%. So, when you have a large enough data set, say thousands of troubleshooting test cases, you will get approximately 80% successful troubleshooting outcomes.

But how many successes should you expect if you just sampled 30, 100, 300 or 1000 test cases? The following table gives you the answer:

You could monitor the accuracy of your Guided Troubleshooting Solution by periodically sampling N test cases and counting how many of those are successful. How to interpret these results?

1. 80% success rate does not mean that every 5th case will be unsolved. When you look at small enough data set, expect to see large enough variation, and that is no reason for alarm. Even if you see 5 successes in 10 trials, that still does not prove you are not going to have 80% success rate. Gather more data, maybe you just had a bad day!
2. Whenever you get successes outside these bounds, do take a closer look. There is still a small chance that there is nothing wrong. But may be there is something else going on. Has your target system changed, and models need to be updated? Is this a new operating condition? Are the FSEs getting confused by the instructions? Whatever it is, it is worth a second look.
3. Do track the moving average. If your average is falling and you are consistently getting worst case results, chances are you are falling behind on your Success rates. On the other hand, if you are consistently beating the average, may be you are overachieving!
4. These tests are known to be too insensitive to small variations in the underlying process, and are therefore suited to measuring the performance of a Global Field Service Organization. However, you can add a rule of thumb or two  to improve its sensitivity. For example
   • If you get two or three back to back data sets that are more than two-thirds of the way to the best or worst case, or four or five back to back data sets that are more than one-thirds away from the expected value, you may want to take a closer look at the underlying data.
   • If you get 8 or more data sets that are all above or below the expected value, it may be time to reevaluate S.

Hope this helps you in measuring success in your Field Service Organization. Let me know how it works out.

I am often asked by a prospective customer this simple question – do you do Prognosis?

If the customer is in DoD/Aerospace world, or has an established R&D and Health Management program, the short answer is yes!

However, customers asking this question often are field service organizations that maintain expensive assets and are looking for an alternative to the traditional break-fix service model. The term Prognosis has become quite popular over the years, thanks to millions of dollars invested by the US Department of Defense in programs such as the Joint Strike Fighter. The promise of Prognostics is simple — wouldn’t it be nice if you could predict how much life each component has left, so that you could replace them just before they failed? But is Prognosis the right tool for you?

First, how good is your Diagnosis? If you are struggling with faults that have already happened, chances are you won’t do any better with faults that have not happened yet! Think of Prognosis as something that involves Diagnosing impending failures and predicting when they will develop into full blown faults. So, Diagnosis is the foundation for Prognosis, and the need for predictive capability makes Prognosis a powerful but expensive technique that should be used wisely only where it is necessary.

This brings us the second question: do you really need Prognosis? For example, your car has two headlights. While it is no fun driving in the rain with one headlight (have you noticed how headlights always seem to fail on rainy days?), having two headlights means that you can still get back home when one has failed. So, redundancy and fault-management are effective ways of reducing unscheduled downtime. For your critical components, evaluate what is the most cost-effective method to avoid disruption in service, and choose wisely.

And now, the all important third question: why do you want Prognosis?

Some people will answer I want to use less maintenance. For example, you may be used to changing the oil in your car every 3 months (schedule-based maintenance) or 3000 miles (usage-based maintenance), but by monitoring the condition of the oil and the filter, you could change the oil only when needed (condition-based maintenance or CBM). This is a valid application of condition monitoring, although strictly speaking, this is not Prognosis. Also, keep in mind that you may not be able to extend maintenance interval for safety critical components without exposing yourself to more liability.

However, most of our customers answer they need prognosis to reduce unscheduled downtime by doing preemptive repairs. Here too, Prognosis is not the only answer.

Let’s take an example — supposing you want to avoid being stranded on the highway due to tire failures. You could add sophisticated techniques that monitor the tread of the tires and how it is wearing out, how the underlying structure of the tire is holding up, the stress on the tire, etc, and predict when failure is imminent. You could develop such Prognosis at significant R&D expense, or, you could simply replace the tires when they look worn (CBM) (e.g., cracks on sidewall and/or tread-depth of 1/32nd inch or less). The second method may cause you to use at most one extra set of tires over the life of the car since you will throwaway tires with still some useful life left on it, but newer tires also improves your safety and performances, which has its own reward. Best of all, you can use the second technique on your current installed base without having to develop new technology.

Let’s also not forget, Prognosis or CBM does not completely prevent unscheduled downtime. You could still hit a pothole and get a flat tire – no matter how new your tire is!

To sum up, there is more than one way to reduce unscheduled downtime: Prognosis, Condition Based Maintenance, redundancy and fault tolerance and fault management. QSI can help you implement all of the techniques discussed here in a balanced health management solution. Health Management is not just Diagnosis or Prognosis, but an effectively engineered delivery of uptime at a reasonable cost.

Let us help you find the right balance of techniques to achieve your objectives.

2020 is the year circled on NASA’s calendar marking the goal that engineers and scientists involved in the Constellation Program have set for themselves to transport astronauts back to the moon.

NASA formed the Constellation Program specifically to establish a base there, and to lay the foundation for eventual explorations of Mars and the solar system before the end of the first half of this century. Much has already been learned from NASA’s previous spaceflight programs, beginning with Mercury and Gemini, and continuing through current projects with the Space Shuttle and the International Space Station.  The Constellation Program, however, focuses on a new generation of spacecraft for human spaceflight, consisting primarily of the Ares I and Ares V launch vehicles, the Orion crew capsule, the Earth Departure Stage, and the Altair lunar lander.  These spacecraft will be capable of performing a variety of sophisticated missions ranging from Space Station resupply to lunar landings.

Qualtech Systems, Inc., (QSI), a technology company based in East Hartford, Connecticut, has been intimately involved with the NASA Space Program since the mid-1990s. QSI’s team has worked closely with engineers and scientists at NASA’s Ames Research Center on designs for the program’s evolving launch vehicles, ranging from reusable launch vehicles (RLVs) in the mid-90s to the current Ares launch vehicle, as well as the International Space Station.  QSI develops software that captures the knowledge of how a system fails and how the failures are detected, then uses that knowledge to guide engineers to make troubleshooting and real-time diagnosis more efficient, capabilities essential to the design of any crew launch vehicle. QSI’s software suite includes three tools used by NASA specifically for the Ares launch vehicles and Orion crew modules. These tools support systems engineering; systems design and testability; automated diagnostics and troubleshooting; and system autonomy. The first, TEAMS-Designer® (Testability Engineering and Maintenance System), is a tool used in design/analysis phases of complex systems. TEAMS-RT© is a real-time diagnostic engine that provides diagnostic functionality for integrated vehicle health systems on board a flight vehicle or embedded into a run-time architecture.  Finally, TEAMS-RDS™ (Remote Diagnosis Server) is an application that can support multiple simultaneous diagnostic sessions from a variety of remote systems.  It was for this software suite, developed during a seven-year collaboration with ARC researchers, that QSI was selected for NASA’s prestigious Space Act Award in 2002.

This exceptional level of technical expertise, combined with the company’s responsiveness and adaptability to NASA’s needs, are among the many reasons why Dr. Ann Patterson-Hine, Tech Area Lead for Discovery and System Health, and the technical contact for the projects QSI has done with NASA since ’93-’94, enjoys working with the QSI team.  She notes that the NASA and QSI teams have developed a genuine collaboration that often alters the outcome of a project and that Qualtech’s technology sometimes materially changes the way NASA proceeds with its designs.  NASA’s approach isn’t simply to purchase a software product and apply it to their designs: they like the ability to test, analyze and modify as they go along.  NASA quickly realized that they could get analysis results out of QSI’s software, (modifying, for example, the placement of sensors based on TEAMS® analysis) and thereby use the tools to improve their designs.

QSI’s willingness to adapt to NASA’s requirements has resulted in some exciting developments in critical systems engineering efforts in the design phases of the Ares launch vehicle project. Working with Crew Launch Vehicle Fault Detection, Diagnostics and Response (FDDR) team members at Ames Research Center and Marshall Space Flight Center, QSI began to enhance their original diagnostic software, actually building NASA’s requirements into their toolset. According to Dr. Patterson-Hine, “The responsiveness of the QSI team enabled us to define new features that we needed in the TEAMS-Designer® tool to support development of launch abort algorithms and QSI rapidly implemented the new functionality to support our project schedule.”

As a result, the systems engineering analysis capability has been significantly improved.  Initially, FDDR engineers were building functional fault analysis models of vehicle subsystems that would ultimately support the design of the abort logic for the new launch vehicle.  As details were added and reviews were held, the subsystems engineers began asking for QSI’s tools so that they could utilize the various analysis features of TEAMS-Designer® themselves in the design of the subsystems.  As timing analysis and effects analysis were added, a TEAMS-NASA version emerged.  Due to the popularity and widespread benefits of these capabilities, QSI merged TEAMS-NASA with their commercial product and have recently released TEAMS-Designer® 11.0.  According to Dr. Somnath Deb, President and Chief Technical Officer for QSI, “QSI draws on its customers’ needs to select the feature sets for its tools.  It is a win-win situation: the customer gets features that are important, and QSI gets to refine the new concept with a real-life application.  This new functionality forms the FMECA-module of TEAMS-Designer® 11.0, and other Aerospace customers, such as Gulfstream, are already benefiting from these.”

Now, multiple centers are using the QSI technology. Ames Research Center, Marshall Space Flight Center, and Johnson Space Center all have TEAMS-Designer® site licenses and several other centers are currently purchasing single-user licenses to support their Constellation tasks.  The innovative modeling and analysis capabilities provided by the QSI tool suite enable design information to be utilized during later system lifecycle phases, such as in ground processing at the launch site and on board the vehicle itself.  QSI’s Remote Diagnostic Server is slated for use in the Engineering Technology Development Program (ETDP) ground systems automation project which is a joint effort between Ames Research Center and other key NASA centers.  In addition, Honeywell, the Orion avionics subcontractor, has selected TEAMS-RT® for on-board Vehicle Health Determination.

Another aspect that Dr. Patterson-Hine likes about working with QSI is that their methodology fits right in with NASA’s goals.  While they are using this technology specifically for Project Constellation, NASA is also developing a process that future project managers can use to ensure that there is greater consistency throughout the project lifecycle.  According to Dr. Patterson-Hine, Qualtech’s methodologies are part of this “blueprint”. “NASA continually works to improve its systems engineering practices and the QSI approach, starting with design assessments and utilizing the same basic model throughout the development and implementation of run-time diagnostic systems and into operations and maintenance phases, enables system health management practices to be integrated seamlessly with conventional systems engineering processes.”

According to Dr. Deb, many organizations, not just NASA, are embracing a systems-oriented approach to the design process. He notes that, “Tools that can reduce unnecessary tests provide cost effective testing procedures, and which are built to work seamlessly in an integrated design environment can yield substantial cost savings over the system’s lifetime.  Markets for such design tools include large prime contractors and government organizations that must integrate numerous subsystems and study testability at the system level.”  He adds, “The TEAMS® tools can be used as training aids for field maintenance engineers and/or diagnosticians. Most significantly, Qualtech’s “Remote Diagnostic Server (RDS)” or TEAMS-RDS® provides these capabilities from a central server computer to thin clients (PC with a web browser) over the Internet, anywhere in the world. The e-commerce potential of this solution has very large (and rapidly expanding) commercial potential. The Qualtech Integrated Toolset is unique in that it provides a comprehensive solution for cradle-to-grave supportability of complex systems whether they are new development or legacy platforms.”

Qualtech was just recently selected a second time for NASA’s prestigious Space Act Award, which highlights “significant and technical contributions to aeronautical, commercialization, technology transfer and space goals” and is given for mature technologies that are proven in NASA applications. The 2008 award was announced in early May, citing QSI’s technologies used by Project Constellation, specifically TEAMS®RT and its relation to real-time diagnosis of dynamic systems.

The beauty of the TEAMS® software is that while it can be used in the complex applications of NASA’s research projects, it functions in essentially the same way for the simpler applications required by other industries and companies, especially those who, like QSI, work closely with NASA.  The idea is to help companies capture information about how their equipment might fail so they can keep it working. The software improves fault isolation, eliminates shotgun maintenance, and increases equipment availability.  What works so well for NASA can work equally well, even at a less complex level, for field service organizations with high value business critical assets, where downtime is expensive.  Orbotech, for example, a company that designs, develops, manufactures, markets and services automated optical inspection (AOI) systems and imaging solutions for PCB production, has deployed the TEAMS® solution worldwide for their entire field service workforce to minimize troubleshooting time and eliminate false pulls. The advanced reasoning of the software enables every technician to perform like an expert, and the solution has paid for itself many times over in the last few years.  KLA-Tencor, a leading supplier of yield enhancement equipment to IC manufacturing lines, acquired the TEAMS® solution after an extensive evaluation showed that a novice technician equipped with the TEAMS® guided troubleshooting solution (GTS) could routinely outperform an expert in troubleshooting a complex system. Their entire field service workforce (about 1000 technicians) is now using the TEAMS GTS.  Other companies that utilize Qualtech software include Lockheed Martin, Honeywell, BAE and General Motors, making it an integral part of a number of critical, innovative projects that receive international attention.

As Dr. Deb likes to point out, these companies, and the customers they serve, don’t always stop to consider how the collaboration between NASA and QSI benefits them in terms of efficiencies and cost savings.  “The technology that works so effectively in a space station environment can be adapted to work just as well on an oil rig or even in a kitchen.  Software has to be of the highest quality and innovation to meet NASA’s needs.  Historically, technology developed for NASA requirements has been prohibitively expensive for any down-to-earth application.  In this case, we have taken a commercial off-the-shelf technology, the TEAMS® toolset, and raised it to NASA standards, without raising the price.  The net result is that we now have a very sophisticated software toolset that companies can utilize to save money.”  He adds, “The same technology that allows NASA to make complex systems easy to maintain, can now help any technician troubleshoot like an expert and fix problems right the first time – be it business jets or automobiles or submarines; and from million dollar semi-conductor fab equipment to affordable kitchen appliances – if it is hard to troubleshoot, our TEAMS® toolset can help service organizations to save money!”

For more information visit http://www.teamqsi.com