Data & Analysis Center for Software

The mission of DACS, like the other IACs in the DTIC IAC Program, is:

To improve the productivity of researchers, engineers, and program managers in the Defense research, development, and acquisition communities by collecting, analyzing, synthesizing, and disseminating worldwide scientific and technical information in clearly defined, specialized fields or subject areas. The IACs' secondary mission is to promote standardization within their respective fields. They accomplish these missions by providing in-depth analysis services and creating products. IACs respond to technical inquiries; prepare state-of-the-art reports, handbooks, and databooks; perform technology assessments; and support exchanges of information among scientists, engineers, and practitioners of various disciplines within the scope of the IAC.

— DTIC IAC Program web site[1], July 28, 2008

DACS receives its authority to operate from the following DoD Directives and Instructions, which constitute the IAC Charter.

The IACs are government organizations regulated by DoD Directive 3200.12; DoD Scientific Technical Information (STI) Program (STIP), dated 11 February 1998; and DoD Instruction 3200.14, Principles and Operational Parameters of the DoD Scientific and Technical Information Program, dated 13 May 1997. Policy oversight of the IACs is provided by the Office of the Secretary of Defense (OSD), Director of Defense Research and Engineering (DDRE). Administrative and operational management is provided by DTIC. Technical management is provided by an appointed Contracting Officer's Representative (COR) from each of the technical host organizations.

— DTIC IAC Program web site[2], July 28, 2008

The DACS was established in the late 1970s. The software community, at that time, recognized the need for data and information about computer software, its development process and the software technology area in general. The rapid expansion of software engineering technology and the proliferation of tools and techniques made it difficult for an individual or organization to remain cognizant of the current status of the software engineering field. This situation resulted in the duplication of efforts in software research and seriously hampered the transfer of technology from the software research environment to the user sector of the software community.

Recognizing the need for an information analysis center to serve the government, industrial, and university community as a focal point for software development and experience data, in August 1978 the Rome Air Development Center (RADC), which is now called the Air Force Research Laboratory (AFRL), contracted with IIT Research Institute (IITRI) to design such a center, named, The Data and Analysis Center for Software (DACS), that would acquire, analyze, synthesize, and disseminate information on software engineering technology.

The decision was based in part on the fact that the DoD and other Federal Agencies found that the establishment of special purpose information analysis centers and technology transfer programs were effective in overcoming problems in technology implementation and diffusion of mission-oriented developments. NASA was cited as an example for demonstrating a benefit-to-cost ratio in excess of ten to one for its Technology Utilization Program. At that time the DoD had already been successful in operating Information Analysis Centers for Metals, Ceramics, Hardware Reliability, and Machineability data.[3]

DACS initially functioned as the DoD librarian for software related topics. The objectives were to:

• Encourage the diffusion of technology to DOD, Civil Agencies, Government contractors, etc.
• Bring about higher levels of utilization of project results in a cost-effective manner
• Increase the productivity and quality of computer software by improving the transfer of software engineering technology
• Assist in diffusing new technology throughout the U. S. industrial base, thereby expanding its capability and competitive posture.
• Provide scientific and technical information analysis services to DoD, Civil Agencies, government contractors, and the private sector in areas relating to software technology needs, developments and trends
• Minimize duplication of effort thereby reducing costs

Since its inception, DACS has been operated by various Defense contractors over the years and the library, and other resources constituting the DACS collection, have migrated to each contractor in turn. The objectives are essentially the same but the terminology used to define them has evolved as the software environment has changed. The means for improving the 'transfer of software engineering technology' now centers around use of web technology to deliver information to the mass community in electronic format, organized in ways that make it easy for the community to acquire and use the information.

DTIC and the DTIC IAC program are notable. The DACS is an IAC. Some DACS products are gaining recognition.[4]

Some DACS reports have been published in peer-reviewed journals and collections.[5][6] Some DACS work is referenced in the professional literature. For example, Warren Harrison et al.[7] and Raffo et al.[8] cite both Vienneau[6] and Thomas McGibbon,[9] the DACS Program Manager. Rini van Solingen[10] cites McGibbon.[11] David F. Rico[12] acknowledges Thomas McGibbon for "inspirational" work that has "been critical to the formation of concepts in this book". Hossam A. Gabbar[13] and Almstrum et al.[14] cite Vienneau.[15] Michael R. Lyu[16] cites a DACS implementation of the Goel-Okumoto software reliability model, distributed by the DACS at the time.

Additional evidence of the effectiveness and outreach of DACS products and services is contained in a DTIC Success Story[17] which describes the significance of a DACS core effort in providing value-added analysis in its synopsis and overview of a General Accounting Office (GAO) report,[18] resulting in the subsequent use of that material by members of the defense community and software industry, as well as extending the usefulness and outreach of the original GAO report in both time and scope beyond two years. The DACS article providing the synopsis of the GAO report was published in the July 2005 issue of the DACS Software Tech News[19]

The primary means of dissemination is via the DACS web site.[20] There, DACS presents information organized around the key software related research areas or topics. Each research area is further organized into content-specific subtopics and information categories. Visitors to a topic area can navigate among the categories to get the type of information desired. Information categories include (but are not limited to) literature, best practices, case studies and lessons learned, education and training resources, subject matter experts, glossaries and acronyms, blogs and other interactive venues for software topics, programs and organizations, service providers and consultants, tools, and other related resources. Subtopics provide greater coverage of the main topic as well as addressing a specific perspective, enabling the visitor to quickly navigate to the desired level of detail.

Some of the research topics included on the web site are: Software Acquisition, Software Architecture, Agile Software Development, Software Best Practices, Collaborative Software Engineering, Configuration Management,

From the web site registered DACS users can search DACS databases, including:

• Software Engineering Bibliographic Database—with more than 350,000 entries, representing the entire DACS collection since its inception. Searching returns bibliographic data, including where the artifact can be acquired. DACS adds to this collection on a continuing basis as resources emerge.
• Software Life Cycle Empirical Database—established to support the acquisition, maintenance, and dissemination of software empirical life cycle data for research purposes and to support the improvement of the software development process.
• DoD Acronym database—contains nearly 30,000 acronyms that relate to software technology and programs of interest to the DoD. This database is continuously updated.
• DACS ROI Dashbord—a collection of information that addresses the Return-On-Investment from Software Process Improvement. Data is collected from open literature and from collaboration with organizations involved with software process improvement such as the Software Engineering Institute.

DACS publishes a quarterly technical journal, called The Software Tech News,[21] which is distributed in printed and electronic format to registered DACS subscribers free of charge and also available for viewing and download from the DACS web site.[20] Authors are solicited for their expertise or experience relative to a chosen theme. Recent themes have included:

• Future Directions in Software Engineering
• Grid Computing
• Net-Centric Software Architecture
• Open Source
• Performance Results from CMMI Based Process Improvement
• Service Oriented Architecture
• Software Archaeology
• Software Reliability Engineering

DACS periodically produces technical reports on a variety of software engineering topics. In keeping with the DACS mission, most reports are made available to the software engineering community as free PDF downloads from the DACS web site.[22] These documents are reporting on current research in the subject and are typically authored by subject matter experts. Some recent reports include:

• Modern Tools to Support DoD Software Intensive System of Systems Cost Estimation: A DACS State of the Art Report, August 2007[23]
• A Business Case for Software Process Improvement (2007 Update): Measuring Return on Investment from Software Engineering, September 2007[24]

The software domain is not an isolated entity. There are many instances of overlap of software related subject matter with the domains of other IACs such as reliability, information assurance and modeling and simulation. For example, software reliability is a subtopic of the broader topic of system reliability. Software assurance is intertwined with the subject of Information Assurance. Software technology is a significant part of modeling and simulation. Because of this natural overlap, DACS has collaborated with other IACs to jointly produce the following documents:

• System Reliability Toolkit:[25] an 865-page document published in 2005, which is the result of a collaborative effort by DACS and the Reliability Information Analysis Center (RIAC), to produce a document that addresses (in addition to hardware) the software and human factors that have an ever more significant impact on system reliability.
• Software Security Assurance State-of-the-Art Report (SOAR):[26] published in July 2007, this SOAR (400 pages) was co-authored by technical experts from DACS and the Information Assurance Technology Analysis Center (IATAC). It provides an overview of the current state of the environment in which defense and national security software must operate, then surveys current and emerging activities and organizations involved in promoting various aspects of software security assurance. It also provides noteworthy trends in software security assurance as a discipline.

As part of its outreach to address the needs of the software community, DACS provides a technical inquiry service, wherein it provides up to four hours of research, free of charge, to address any technical inquiry from any member of the DACS community. In some cases, DACS connects the inquirer directly with a Subject Matter Expert (SME) to satisfy the request. DTIC, the organization that funds the DACS, regards the number of technical inquiries handled by DACS each month as a key performance metric for operation of the IAC. The scope of questions asked is very broad, ranging from simple questions asking for a definition of an acronym, to very complex questions. The following list contains some recent requests that convey the essence and scope of this service.

• How does Agile handle turn-over in developers, vendors, etc.? My question stems from the fact that a system may be developed by one vendor, but may need to be maintained by another.
• "I've noticed a tendency for acronyms to proliferate within DoD. Is there a DoD clearinghouse/arbitrator through whom these entries are filtered?"
• We are implementing a new hr system - I have been tasked with proposing a model through which modifications/customizations standard ERP software can be evaluated and quantified against the objectives. Any recommendations on approach - thanks
• I am looking for information on how DoD manages software development project so there is no duplication.
• How does Agile implement Earned Value?
• The patron is looking for, size estimates of logical source lines of code ratios for different languages. He has looked at Capers Jones but needs more information.
• "I am trying to ferret out some very specific information about monitoring software defect trends. Again, it was years ago but I was a software development Program Manager and my team used a tool that tracked the progress of a defect find/fix rate over time. This trend was called a "glide slope". It was not a crystal ball but was reasonably illustrative in forecasting code stability and release date. I cannot locate my copy of the algorithm and have been doing an Internet search with no luck. Thanks for any help that you can provide."
• "Is there a standard methodology for determining the availability of a software system?" Under a DoD effort, we are trying to define a methodology that is used to derive an availability number.
• "What's your experience with model based testing. What are the drawbacks?"
• Student asked for an explanation of function points as a measure of software size, and whether they are useful for scientific programs.

As is the case with all DTIC managed IACs, DACS serves as a contract vehicle that enables DoD components to acquire technical services from commercial providers (Defense contractors, small businesses, etc.) and academic institutions without having to engage in a formal competitive bid process. This saves time and resources for the government and enables awarding of a contract within a four to six week window. This IAC technical area task (TAT) service has the following characteristics which are beneficial to the government:

• Access to competed single-award contracts with no Request For Proposal (RFP) process
• Large and scalable delivery orders with size and scope determined by the acquirer
• Long periods of performance (up to three years) - which tends to reduce the cost of procurement
• Incremental funding managed by the acquirer
• Collaborative work plan/statement of work (SOW)development process
• Skilled program managers with established processes and procedures
• Deliverables-based contracts that share scientific and technical information across the community of interest - thus supporting cost savings through reuse

The efficiency of the TAT contracting process and its bypassing of a formal RFP process is premised on the business model of the DTIC IAC program. Specifically, contractors must compete for the opportunity to operate an IAC. This is a full open competition, and award is based on the capability of the contractor to perform software-aligned technical areas tasks, as well as to perform the core functions of maintaining software-focused libraries and developing a software center of excellence. When bidding on an IAC, contractors form collaborative teams to ensure the best overall coverage of the domain. Therefore, the contractor operating the DACS has already competed for the right for their team to perform TATs. The DACS contractor manages all TATs, insuring that any scientific and technical information (STI) resulting from the TAT is added to the DACS collection for dissemination in appropriate venues. When the government wishes to acquire the services of someone other than the DACS contractor, it is done as a TAT based on a subcontracting arrangement between the service provider (team member or other) and the DACS contractor. This business model gives the government a great deal of flexibility in working with DACS to acquire the services from the most qualified organizations. This is especially true in scenarios where the technical expertise needed resides within a small group in an academic institution, or a small business.

TATs cover a broad spectrum of activities within the software domain. The list below is included to communicate the range and types of activities performed under TATs.

• ACE program - Advanced Course in Engineering (ACE) Cyber Security Boot Camp. The mission of the ACE is to develop ROTC cadets into cyber officers — airmen, warriors, and leaders. The ACE program was developed under a Data and Analysis Center for Software (DACS) Technical Area Task and is the only cyber education offered by the Department of Defense (DoD) for ROTC cadets. The cadets are selected from the best students at U.S. colleges and universities. ACE targets the top students in computer-related disciplines, and it teaches cadets to become original thinkers, problem solvers, and technical leaders.
• Air Force NCES Tactical Experiment (ANTE) - The DISA sponsored Net-Centric Enterprise Services (NCES) program has established an early adopters program and invited participants to enroll in a pilot project to exercise the NCES information services. The DACS is supporting AFRL/IF and the Secretary of the Air Force, Net Centric Enterprise Office, and the Warfighter Integration Office (SAF/XCISS). AFRL/IF and SAF/XCISS are sponsoring experiments that will define NCES capabilities for tactical users. The DACS, under the ANTE effort, is conducting research and development for NCES tactical users with restricted bandwidth. The ANTE effort follows a tactical scenario that illustrates an expected data flow in which a tactical warfighter needs data to complete a mission. The data flow incorporates reach back access through NCES and local access through a tactical enclave.
• Net-Centric Enabled Wide Area Surveillance and Information Dissemination (Project Marti) - a Net-Centric research program funded by the Air Force Research Laboratory Information Directorate, Rome, NY. The DACS, in conjunction with the Boeing Phantom Works and the U.S. Air Force have demonstrated for the first time how, with advanced airborne networking and information management technology, a near-space vehicle can be used as a flexible, low-cost, theater-wide information broker that provides real-time tactical information to ground forces to enhance their effectiveness and survivability.

This initiative, started in 2003, is focused on providing the software acquisition and software engineering communities with detailed information about prevalent software acquisition and development best practices that may have a positive impact on program risks and ROI. DACS is conducting extensive research on each identified practice and then publishing a technical document for each practice, that contains a detailed description of the practice or practice set, including the interrelationships among practices and an extensive lists of resources relating to that practice. The documents are available for viewing and download, free of charge, from the DACS Gold Practice web site. The document also identifies relevant web sites, tools and methods, experts, training opportunities, literature, terms, practice origins and recommending origins.

The initial set of practices which DACS has targeted for this initiative were first identified in the PhD dissertation of Dr. Richard Turner titled “Implementation of Best Practices in U.S. Department of Defense Software-Intensive System Acquisitions”.[27]

The following practices can be downloaded from the DACS Gold Practice site:

• Acquisition Process Improvement
• Architecture-First Approach
• Assess Reuse Risks and Costs
• Binary Quality Gates at the Inch Pebble Level
• Commercial Specifications and Standards/Open Systems
• Insure Interoperability
• Formal Inspections
• Formal Risk Management
• Goal-Question-Metric (GQM) Approach
• Integrated Product and Process Development (IPPD)
• Manage Requirements
• Metrics-Based Scheduling
• Model Based Testing
• Plan for Technology Insertion
• Requirements Trade-off/Negotiation
• Statistical Process Control
• Track Earned Value

This initiative has been ongoing since the late 1990s. Software Process Improvement (SPI) has received much attention in recent years; however, it has been very difficult to translate benefits achieved in one organization to another organization. This initiative is focused on accumulating data from SPI efforts, analyzing it, generalizing and modeling the cost benefits one can achieve from SPI efforts. In 1999 DACS produced a State of the Art Report (SOAR) titled "A Business Case for Software Process Improvement" and revised this report in 2007[28] to provide new insights into the details necessary to demonstrate from a business perspective the benefits of improved software management using SPI techniques. This revised SOAR examines the business implications of some more recent SPI practices, including the Capability Maturity Model Integration (CMMI), agile development, and systems engineering.

The DACS has recently implemented a new capability called the "ROI Dashboard©", on the DACS web site to provide updated information about return-on-investment (ROI) results. The ROI Dashboard© also contains updated information for practices such as inspections, reuse, and secondary benefits. It establishes a framework whereby the current methods of performing software development can be compared to any proposed improvements, graphically displaying open and publicly available data and providing standard statistical analysis of the data.[29]

The Total Electronic Migration System is a project funded by DTIC to support the collection of scientific and technical information held in the IAC document collections and the subsequent dissemination of that information to a restricted DoD community through a sophisticated search capability. DACS, along with the other DTIC managed Information Analysis Centers, is collaborating with the TEMS initiative, by transforming to electronic format the documents within the DACS collection (DACS library) that exist currently only in printed format and uploading them to the database created by DTIC to support the TEMS effort. In many cases, the IACs are the single holders of valuable documents developed prior to 2000. This initiative preserves them and expands their outreach. Each IAC is transforming many thousands of documents to PDF format, making them more accessible to the IAC researchers as well as the TEMS. This initiative is ongoing although many IACs are still working on scanning the backlog of documents that existed in printed form only. For further details visit the TEMS web site and review the 'About TEMS' section.