Understand the government and enjoy success in high level jobs.

We would like to thank the following individuals who assisted us in formulating many of the conclusions contained in this report.

 

Mr. Edward J. Hanley

Director,

Office of Information Resources Management Environmental Protection Agency

 

Mr. Michael Gulley

Deputy Director, Field Operations, Veterans Administration

 

Mr. James Neary

Executive Director, Board of Contract Appeals, General Services Administration

 

Mr. Donald J. Page

Director, Authorization

and Management Reviews Division

Information Resources Management Service General Services Administration

 

Mr. Thomas Gildea

Acting Chief,

Specifications Branch,

Systems Development Division, Department of Commerce

 

Mr. Vidal Falcon Director,

PAID Redesign Project Veterans Administration

Mr. Stanley Polinsky

Program Analyst/Computer Specialist General Services Administration

 

Mr. Robert L. Ross

Director, Agency Liaison Programs, General Services Administration

 

Mr. Douglas R. Posson

Deputy Assistant Director for Inforrnation Systems Information Systems Division

U.S. Geological Survey

 

Mr. John Caron

Assistant Commissioner,

Office of Technical Assistance General Services Administration

The following employees of the Office Of Innovative Systems, Information Resources Management Service, General Services Administration, were responsible for developing this report:

 

Ira A. Penn, Team Leader

 

Eugene F. Brown, Management Analyst

Robert J. Carey, Management Analyst

GSA DC-8911100865

FRANCIS A. McDONOUGH

Deputy Commissioner

Information Resources Management Service U.S. General Services Administration

Preface

Since the mid 1960s the Systems Approach has been the principle methodology used to identify requirements in computer based application systems. This approach was developed to provide a way to identify and coordinate all related requirements and to wrap them up in a single bundle for planning, oversight, design and implementation.

 

The key words are "all related requirements wrapped up in a single bundle." The problem is that at some level, everything is related. Suppose, for example that an agency determines that it needs a new payroll system. The systems approach will uncover requirements related to the personnel inventory, position inventory, and the skills inventory systems. Soon, the agency is led into considering not just a new payroll system but a combined and broadly defined payroll and human resources system. Often an important title is assigned such as "Human Resources Management for the Year 2000." So, what had been a difficult, but doable challenge to develop a new payroll system becomes a very complex, and perhaps not doable Grand Design aggregation of what could be four separate systems.

 

Central agencies such as the Office of Management and Budget and the General Services Administration, Congress, including the General Accounting Office, and agency oversight officials endorse the systems approach and the Grand Designs that result. All requirements are described in a single proposal. There will be no surprises next year or in later years.

 

Grand Designs, building on the Systems Approach, are second nature to all of us in key positions in the government. They are second nature also to our contractors. Universities have been teaching this approach for more than 20 years and, we are all products of this era.

 

Looking at recent Federal experience, and looking back over 20 years, there is reason to question whether Grand Designs are the way to go. In reality, there are few examples of success; and, there have been many failures. One reason is that Grand

Designs become so big and costly that they attract an inordinate amount of attention from Congress, including the General Accounting Office, and from oversight officials at all levels in the Executive Branch. As a result, Grand Designs have been hard to sell in the Executive Branch and to Congress. Secondly, these are hard to award because the huge sums of money can make a big difference to winning or losing vendors. Thirdly, when Grand Designs do sell, they are usually compromise solutions that are even more difficult to implement. Finally, Grand Designs often require 10 years to implement after award. During this period, agency managers may change three or four times, vendor managers change at least as frequently; and, the terms of the contract and the work to be done change so much that after several years nobody knows what was being sought in the original specifications. In many ways, nobody is responsible in Grand Designs. And, it is impossible to hold a contractor accountable.

 

Year after year, Federal managers follow the Grand Design approach despite the poor track record. One reason is that there is always a new fad on the scene that diverts attention from the problems with the

approach. In the early 1960s, the Program Evaluation and Review Technique (PERT) was the vaunted way to manage big, complex programs. In the 1970s, planning was emphasized as the cure-all. Today, in 1988, the current fad is to bundle all the requirements together and hire a Systems Integrator.

 

The question is whether anything can make Grand Designs work. What is the consensus among a sample of experienced senior Federal managers? Should the government begin to downsize its major systems? If so, how can it be done? This report begins to answer these questions.

Background

At the 1986 Partnership in Administration Conference, in Reston, Virginia, and at the Information Resources Management Conference (IRMCO) in Richmond, that same year, GSA was asked by agencies to conduct a study to determine if the Grand Design approach adds to the difficulties

of implementing major applications systems in the Federal government. A Grand Design refers to a large computer application system in which all possible requirements are aggregated. Grand Design encompasses all phases of system development, including planning, design, procurement, installation, and evaluation. It includes hardware, software, telecommunications, and other services and technologies appropriate to large, complex systems. This report outlines our findings about the Grand Design approach. Agency experiences and recommendations for future actions are provided.

Study Method

To conduct this study, it was necessary to review the issues from a variety of viewpoints. Twenty-one individuals were interviewed during the early stages. They were directly involved with the installation and design or procurement of ten systems in the Grand Design category. The individuals interviewed included systems managers, procurement specialists, policy officials, and vendors.

 

Also, much has been written which borders on the subject. Fifty articles and published reports were studied to provide further background. A selected bibliography is included following the appendix section.

 

As the interviews progressed, the problems and concerns articulated began to fall into 19 categories. Therefore, the interviewees were asked to rank the importance of each category. The ranked findings are summarized in the Matrix of Findings section. Finally, a series of recommendations is provided relative to the findings.

Findings

Many concerns surfaced during the study. The most significant ones are described below.

 Regulations, Procedures, and Personnel

There are many regulations in existence covering IRM acquisitions. Most line managers at the operating level find these regulations difficult to

understand. In addition many agencies write their own directives to describe the environment unique to the agency. It is not unusual for a procurement official to be faced with the need to satisfy the requirements of six different Federal organizations when acquiring information resources.

 

The overall impact of Government contracting is severe when compared with the private sector. Sole-source contracts are a virtual impossibility. Many agencies indicated in the survey that the red tape associated with a major acquisition is overwhelming.

 

Persons and organizations involved with a typical, large ADP acquisition include users, ADP personnel, procurement officials, legal staff, central agencies (GSA, GAO, OMB), Congress and its staff, and vendors. Often, these individuals and groups do not have a complete understanding of the requirements, procedures, and regulations under which the agencies must function.

 

The various personnel in the acquisition cycle frequently blame each other when things are not going their way. Inadequate review and lack of priorities set by management on procurement requests results in procurement office backlogs that inhibit progress on large critical projects. Communications between persons involved in the procurement phases frequently is inadequate.

 

Grand Design acquisitions are made only once or twice during the career of a single individual. Additionally, one does not learn to design major systems in school. There is a strong element of "learning by doing" in major Federal systems. For an agency’s most critical applications, procurement personnel are not always involved at the inception of a Grand Design project and therefore have little understanding about its importance.

 

In several instances, individuals who were not suited for the responsibility were put in charge of Grand Design projects. Frequently they did not have the right experience and background and could not or would not, make decisions. Even when they made a decision, they often did not have the authority to make it stick.

Those interviewed thought that top management often did not understand some of the problems surrounding Grand Design concepts, especially the time that it takes to get the system "on line." Many top managers stay in their jobs for only about two years, which is not long enough to be other than a disruption in long-term complex systems. Some top managers were unable to distinguish the difference between a conventional computer system that is designed to do one thing, and other systems, such as revenue accounting, that may never do the same thing twice because the programs are being continually changed by law and other forces. Others believe that top management simply does not want to get involved or is not interested and that the lack of support from the top makes it difficult to succeed with long-term, costly, Grand Designs.

 

Most agencies voiced considerable concern over staffing in the categories of planning, procurement, and of course, operations. There were a plethora of gripes: the GrammRudman-Hollings Deficit Reduction Act; hiring freezes; retirements; inability to pay well or promote; lack of personnel slots; using personnel who were often incompetent, union problems, and so on.

 Contractor Protests

Before 1984, ADP contractors presented their protests to the General Accounting Office (GAO). Reaching a resolution on these protests took a long time and GAOs rulings were usually in the Government’s favor. Understandably, such rulings displeased the contractors.

 

The Competition in Contracting Act (CICA) (P.L. 98-369) was enacted on April 18, 1984 and GSAs Board of Contract Appeals (GSBCA) was given additional responsibilities to hear protests concerning ADP acquisition. CICA also left the GAO protest process intact. Now there are two bodies to resolve contract protests, GSA and GAO. Although the GSBCA maintains that its decisions are evenly split between the vendors and the agencies, the perception in agencies is that GSBCA favors the contractors while GAO favors the Government.

In 1985 the GSBCA received 89 protests; in 1986 it received 197; and in 1987 the count was 262. A GSBCA official stated "vendors are extremely pleased over the quick turnaround time that the Board gives." CICA requires GSBCA to dispose of protests within 45 workdays and this requirement is almost always met.

 

Agencies believe that most contractors do not normally protest; but some protests appear to agencies to be frivolous and lodged for a variety of motives. A contractor might want to buy time to respond to a bid; or he might hope that conditions of the proposal might change so that they are more easily met; or to allow time to further develop equipment.

 

Prior to the enactment of CICA, procurement activities could continue while a protest was resolved. Under CICA, however, when a protest is lodged, all activities on that contract may be ordered to stop, depending on the circumstances of the protest.

 

Another complaint was that contractors often make promises that they cannot deliver. Some concern was voiced over 8-a set-aside contractors. Some are not able to deal with the paperwork and are unable to understand the regulations.

 Planning

Agencies usually do not have an overall program plan that shows where the agency is going and what it expects to achieve next year, or for the long term. However, agencies are required to do IRM support planning on a five-year basis. Because IRM is a mission support function, the preparation of detailed technology plans to support unplanned programs is questioned in many sectors of the government.

 

Conversely, the need for planning for specific systems development and implementation efforts was stressed by most interviewees. It was also stressed that such planning is not being done ... especially for Grand Design projects.

 

For example, an agency designing a large time, attendance, and payroll system had to redo much of

its design work halfway through the project because the General Accounting Office issued new reporting procedures. If planning had included coordination with GAO, the problem might not have

materialized. Similarly, this same agency did not consider all the different pay scale systems in effect. If an agency financial management official had been involved in the original planning then this problem too might have been avoided.

 

Lack of coordination among management, users of the system, procurement personnel, and other key players during the planning phase can be a big problem according to survey participants. The reason is that at this stage everyone should know the whole picture. They should be aware of the requirements associated with a "Grand Design" project. They should also know the limitations and responsibilities of the other personnel involved in a Grand Design endeavor. Lack of coordination and failure to bring in all parties can lead to disapproval for the system from any quarter.

 

A widespread concern is the lack of life-cycle planning for computer equipment by ADP managers. The most important point to be made is that the acquisition cycle should be started well in advance of the end of the life cycle of the

equipment. If proper planning is done, the procurement process for replacement should be completed before equipment has reached its life expectancy. In other words, the new equipment should be already installed when the old equipment reaches the end of its economic and technological life.

 Placement of the Program

The study team found, more often than not, that the responsibility for a Grand Design project was placed too low in the organizational structure to be effective. This created difficult communication problems, as system managers were unable to reach top management for guidance and decisions. In several instances special communication channels were established that bypassed the normal chain of command. The jobs got done with this arrangement, but not without demoralizing side effects.

In one agency a Grand Design project was started and controlled by the agency head. The first phase of the project went smoothly. Because all was going well, the project was assigned to a lower level official. Sharp differences began to develop among the various operating units. Everyone became a boss and some of the units committed themselves to different hardware and software. Decisions by the "leader" of the project were constantly questioned. The project fell several years behind schedule.

 

Lack of adequate top management involvement or support can be fatal to Grand Design efforts. The projects are too large and too broad in size to be delegated to lower levels. Creating "unofficial" leadership positions undermines the organizational hierarchy. Persons running Grand Design projects must be in the ranks of top management or have full access to the top when it is needed.

 Funding

Projects that have high visibility such as the construction of a veterans hospital have funding appropriated at one time to cover a number of years. However, the government funds most systems on a year-to-year basis.

 

If provisions are not made for multiyear funding, agencies must go back to OMB every year to "resell" their projects, often to different people, and sometimes to different administrations. This often made officials fearful that their programs might not be able to continue. Agencies complained of the need to provide five-year plans for projects with one-year budgets. Redundant justification exercises year after year add a great deal of work and risk to Grand Designs.

 Internal/External Pressures

Many factors affect Grand Design efforts. One that evoked considerable response from those interviewed was pressure from within the agency and pressure from other organizations. Although it might be thought that external pressures would be the major problem, internal pressures, within the agency, are every bit as difficult to address. Political appointees are generally perceived as not understanding the problems and procedures involved with large information systems. Appointees are seen as having little concept of the time involved to implement a major system. They are viewed as unrealistically demanding of results "now." Political appointees want to see results while they are in office.

 

One agency tried to establish a system too quickly because of a Congressionally mandated project schedule. The system never has operated as intended. In another case, external pressure required changes to the system design specifications midway through the project. It was thought that the changes would speed things up. They did not.

 

An additional aspect to the "pressure“ problem is that top management is rarely willing to support line managers who encounter difficulties after taking risks to meet the unreasonable time demands levied from the top.

 

The Federal political arena does not tolerate problems or failures. This could promote a no-risk attitude among managers that is entirely inconsistent with that required for Grand Design projects, most of which are inherently high risk and hopefully, high-reward efforts.

 Unrealistic Time Schedules Set by Others

Many problems can develop when arbitrary timetables are set for a system. In one agency Congress mandated an unrealistic date by which an extremely large and expensive system was to be operational. Additionally, there was a requirement for a twenty- year life cycle report. These requirements caused the agency to 44 stampede." Planning was not adequate and only two contractors were responsive to the bid. Personnel were "borrowed" from other operating units to help with the huge endeavor causing other programs to suffer. Many of the personnel were not qualified, leading to

an almost total dependency on the contractor. Appreciable cost overruns developed which led to GAO audits. Several years later, the system was still not operating as intended.

 Problems that Occur When the Project Falls Behind Schedule

If a project falls far behind schedule it runs the risk of being shut down. Interest by top management may flag and funding may be difficult to obtain. If a project is shut down temporarily, it may not be restarted. In major Federal systems, including Grand Designs, the only real alternative when problems occur is to continue to move ahead. Redirection is rarely possible

 Conflict Among Bureaus

Conflict is often a result of little or no coordination. One department reported that the various bureaus wanted to buy different equipment and inappropriate software and had made the commitment to do so. Each bureau thought that it was the department, and each was pulling in a different direction.

 Matrix of Findings

Throughout the study 19 issues surfaced. They are listed below in rank order. During the interviews agency officials were asked, “In a large computer system (Grand Design) how would you rate the importance of the following factors in the planning, procurement, implementation, and operation phase?” Responses were weighted and then averaged. Note: Of those interviewed, no manager experience a problem in every one of the 19 issues listed. However, the numerical ratings that appear in the following matrix are indicative of the importance that the interviewees attached to the subject whether or not a problem was ever experienced.

Conclusion

The Grand Design approach can cause severe problems. Many of the problems are not unique to the Government. Certain airlines, for example, readily admit that if they knew in advance how much their automated ticketing and reservation systems were going to cost, and the difficulties that would be encountered in getting them implemented, they would not have started the projects in the first place. Yet, in some respects, the problems, while not unique, are exacerbated within the Government owing to forces such as fiscal year budgets, Federal regulations, policies from many sources, and internal and external pressures that affect the agencies.

 

A Grand Design implementation requires time ... a good deal of it. It is not unusual for a large system acquisition to require three years, or even longer before award. Implementation may require another ten years after award. The system is vulnerable during acquisition and during the first several years of implementation.

 

Grand Designs cost a lot of money. Consequently they attract a great deal of oversight attention from various congressional committees, the General Accounting Office, and the central agencies. Further, a very large award can, in some cases, make a major difference to most ADP equipment vendors. They fight very hard to win and will use the bid protest process and other techniques in the late stages of a major acquisition.

 

It is critical that knowledgeable and capable staff personnel be available to guide and manage the acquisition and implementation of a large ADP system. If an important member of the technical or management team leaves, or is otherwise incapacitated, the project may fail.

 

Most large projects rely on renewed funding annually for their continuation. If an agency suffers a budgetary cutback that results in an interruption of the project, it is likely that the project will not be restarted and the previous investment will be lost.

Forces, often beyond the control of agencies, are an ever-present fact of Government life. Whether internal or external, there is always pressure on Federal agencies to change priorities. Consequently, long-term, expensive projects in which the benefits are out in the future are likely candidates for cancellation, or cutbacks.

 

Large system designs are not easily changed once commitments have been made. The specifications and requirements of a Government contract cannot always be modified without incurring significant delays and costs. At the same time, the politics, policies, and functions of Government programs are constantly changing. New programs are started and old ones are modified or eliminated. A change of administration bring changes of direction and perhaps changes in system design as well.

 

The Grand Design approach to large systems usually places an agency in a precarious position in this age of rapidly changing technological evolution. There is every possibility that a large system that requires an inordinate amount of time will approach obsolescence by the time it is operational.

 

In summary, there are serious issues to be faced when contemplating a Grand Design effort. Under the best of circumstances the approach is risky. A Grand Design can work, but the odds are against it.

 

In Table 2, the 10 issue areas identified in the survey as having the most effect on Grand Designs are presented.

Table 2
Ten Issue Areas That Have The Most Effect on
Grand Designs

1.         Coordination problems within the agency during the planning phase

 

2.         Procurement problems during the procurement phase

 

3.         Lack of Acquisition skills during the planning and procurement phases

 

4.         Placing the program high enough in the organization in the planning phase

 

5.         Uncertain funding during the planning phase

 

6.         Audits by GAO during planning and procurement phases

 

7.         Problems with contractors during procurement

 

8.         Staffing problems during planning, procurement and operations

 

9.         Problems with procurement regulations during the procurement phase

 

10.     Unrealistic time schedules during the planning phase

Steps that Can Be Taken to
Increase the Success Rate for
Grand Designs

Below are the top 15 items that suggest themselves from the discussion of findings and the 2 tables.

 

Individual Agencies, GSA - Apprise top management of what is involved in the planning, procurement, implementation, and operation of large information systems. Include program officials

in the planning stage to lend expertise on the functions the system should perform. Ensure that procurement personnel are involved with the systems personnel and the users at the inception of the project.

 

Individual Agencies - Place the program at a high enough level, and in the proper place, within the organization and keep it there. Ensure that the necessary authority for all phases of the project lies within the organization where the program is placed.

 

Individual Agencies, GSA - Consider participating in General Services Administration’s ongoing Trail Boss Program. This is an excellent program for top management and key officials involved with Grand Designs. Review the findings and recommendations from General Services Administration’s "Go for 12" program.

 

Individual Agencies, Congress, GSA - Investigate the chances of relaxing some of the more restrictive aspects of the Competition in Contracting Act such as allowing procurements to continue while a

protest is being resolved.

 

Individual Agencies, GSA, OPM -- Present staffing problems to top management. Consider cross training personnel in the procurement cycle. For example, have users take courses in basic procurement management.

 

Individual Agencies - Appoint experienced, decisive project leaders in order to relieve coordination and conflict problems within the agency.

 

Individual Agencies, GAO, OMB -- Work closely with agency, financial managers. Include them in all necessary phases of an acquisition to solicit their advice. Seek multi-year funding on a large project where appropriate. Do not attempt any project if funds are unavailable or insufficient.

 

Individual Agencies, GSA - Initiate a "ground swell" movement to simplify, and make less ambiguous, the Federal Acquisition Regulations. Until then, "the letter of the law" should be followed as closely as possible.

Individual Agencies, GSA - Seek proper instructions for sole source procurement when that method appears to be advantageous for the Government.

 

Individual agencies, any outside delegating authority - Try to 14 educate" those who, any set unrealistic time outside delegating schedules. Explain what is authority really involved and how long the various phases of Grand Design might be expected to take.

 

Agency Official, Contracting Officer, Contractor - Involve the agency contracting officer "up front," and as often as necessary. Ascertain that the contractor knows exactly what is expected of him. Monitor progress, especially where deliverables are concerned

 

Individual Agencies - Use multiple award schedules, and buy "off the shelf hardware and software whenever possible in order to minimize contractor protests, and to simplify the entire acquisition process. If this is not always possible, then ensure strict adherence to the FAR.

 

Agency Official, Financial Officer -- Work closely with the agency official, financial officer during the financial officer planning phase, and thereafter as often as necessary, during a Grand Design acquisition to minimize bid protest and other problems.

 

Individual Agencies - Keep the project on schedule and complete the project as quickly as possible. The longer a project takes, the more vulnerable it is to extraneous forces that can adversely affect the effort. For example, unforeseen changes in personnel brought about by law, such as the Gramm-Rudman-Hollings Deficit Reduction Act; funding problems; changes in regulations; and problems among operating units within the agency.

 

Individual Agencies - Do not place unrealistic reliance on the contractor. Provide back-up personnel in the event that the contractor becomes unavailable.

A Modular, Evolutionary Approach May Be A Preferred Alternative -- A modular, evolutionary approach should be considered when developing and implementing a Grand Design. There are two main tasks:

 Create an environment for modular, evolutionary systems.
 

§         Building a system in a modular, evolutionary way requires first that a skeleton version of a master plan exist. This does not always require detail or precision at first. The detail will be added to the skeleton in an evolutionary manner over time as early modules are implemented and as lessons are learned.

 

§         The modular, evolutionary approach implies that progress will occur systematically over a period of years. At the outset, the path to success will not be known with precision. What is known is that key people will come and go, technology will change, and the rules under which systems are acquired will change as requirements emerge.

 

§           Anticipating these changes, an environment needs to be established at the outset within which the modular system can evolve.

 

§         Modules are by nature smaller. Therefore they can be processed on smaller computers. This may imply the need to decentralize computing resources.

 

§         Architectural, data, and software standards are needed at the outset to provide connectivity to permit the job to proceed over the years as the requirements evolve.

 

§         Portability through the adoption of an operating system standard such as UNIX is a key factor in creating the environment for modular, evolutionary systems.

 

§           Acquisition of equipment resources as a commodity will simplify acquisitions and permit buying equipment when it is needed

and in the amount needed. In turn, this will permit the manager to keep the equipment up-to- date routinely without disruptive major procurements in the middle of the program.

 

§      Finally, a training program will be needed to train replacements for current political appointees, program users, and the implementation team.

 Divide the work to be done by the system into logical packages.
 

Determine the separate objectives of the                     proposed system. Seek logical and

functional break points. When possible, divide the project into sub-parts that address different project objectives.

 

§      Find sub-parts that can be effectively managed by the assigned agency management staff. The project modules must be tailored to the size and ability of the available management staff. Starting with smaller modules may be an effective method of training the management staff for larger more complex modules later.

 

§         Find sub-parts that are logical and functional. Each project module should address a single system or program function.

 

§      Procure sub-parts inexpensively as separate entities. Procuring modules tailored to meet known, near-term needs will decrease the cost of the total system.

 

§         Budget at the sub-part level, rather than at the Grand Design level. Under this

approach, it will normally be easier to obtain approval. Also, since budgets vary from year-to-year, modules should track with the one-year budget authorization.

 

§      The sub-parts will function as separate entities or as extensions of previously implemented modules. Modules should be structured in such a way that even if various

program changes cause the cancellation of future parts of the project, modules already implemented will still be valuable.

 

§         The focus on sub-part modules will make it possible to show tangible results at regular intervals. If new modules can be introduced at regular intervals (every 12 to 18 months for instance) effective progress will be demonstrated to users and management. This demonstration of progress and value will help gain political and budgetary support for the project.

 

§           Bring the system up one module at a time over several months or years until the entire system is operational.

 

§         Correct deficiencies or problems as you go. Problems and deficiencies will be easier to see and correct at the module level rather than at the Grand Design level.

 

§         Add unforeseen, but needed, capabilities into new modules as they are developed. It will be relatively easy to determine the need for, and the usefulness of, additions to the system "as you go."

Summary

We are in an era of accelerating technological change. Because of this, there is a trend toward complexity... and complexity is expensive. Bigger, or more complex, is not necessarily better. The converse may even be true. A modular, incremental approach is highly recommended, if possible, to reduce the project to manageable proportions, keep costs down, and be able to cope with the problems that inevitably occur with Grand Designs.