DHS Source Code Inventory Process (SCIP)

The SCIP comprises three main sub-processes: inventory, open source release and inventory submission. The inventory process involves identifying and recording all DHS custom-developed code and comprises activities including determining usage types and defining metadata values for custom-developed code. The release process involves the transition of inventoried custom-developed code for public open source release to a publicly available repository and comprises activities including identifying appropriate open source licensing and performing security analyses of custom-developed code. Note that during the release process, new information about custom-developed code may be generated, requiring an update of metadata values defined during the inventory process. The inventory submission process involves the verification and submission of the final custom-code inventory documentation.

To comply with DHS Policy Directive 142-04, these three processes must be performed by each DHS Component. Figure 1 shows the three SCIP processes and the relationships between them. Click on a SCIP process for more information.

A summary of the SCIP requirements for DHS Components are shown in Table 1.

Policy Directive 142-04, DHS Reusable and Open Source Software (OSS) Framework, is a policy that defines the Department of Homeland Security (DHS) Office of the Chief Information Officer (OCIO) activities, roles and responsibilities to ensure compliance with Office of Management and Budget (OMB) Memorandum M-16-21 Federal Source Code Policy: Achieving Efficiency, Transparency, and Innovation through Reusable and Open Source Software (herein referred to as The Federal Source Code Policy or OMB M-16-21). OMB M-16-21 requires agencies, when commissioning new custom code, to release at least 20 percent of new custom-developed code as Open Source Software for three years, and to collect data concerning new custom software to measure performance.

DHS OCIO Responsibilities

The DHS Chief Information Officer (CIO) is responsible for developing, maintaining, disseminating, and implementing new policy and instructional requirements of OMB M-16-21. Actions assigned to the DHS CIO include:

• Inventorying, on a continuing basis, all DHS custom-developed code and related information, and publishing a machine-readable source code project inventory that conforms to the JavaScript Object Notation (JSON) schema for cataloging metadata on https://www.code.gov.
• Ensuring that all new information technology (IT) with noncompliant software requires justification, and that the resulting contracts include provisions to ensure delivery of custom-developed code, documentation, and other associated materials from the developer throughout the development process.
• Providing guidance around standard version control systems and a centralized software repository as identified by the OCIO Office of the Chief Technology Officer, allowing for future compliance via automation.
• Annually, beginning Fiscal Year (FY) 2018, DHS will commence planning and coordination to implement the following activities as soon as practicable: o Release a minimum of 20 percent of custom-developed code as Open Source Software each year. The percentage is based on the total number of code projects in the inventory.
  • In coordination with the Office of the Chief Procurement Officer (OCPO) and Office of the General Counsel (OGC), develop and implement a policy or process that requires DHS to apply the three-step Software Solutions Analysis during the procurement phase for new IT systems. When contracting for developer services, DHS encourages vendors to use OSS, open standards, and modular architecture that meets DHS standards for security, federal interoperability, and data integrity wherever possible.
  • In coordination with the Science and Technology Directorate (S&T), OCPO, OGC, and other stakeholders, develop and implement a process that ensures appropriate rights to custom-developed code, recognizing that a vast majority of code produced for DHS is written by contractors. In order to increase the percentage of delivery of source code using free and open licensing, the CIO works with the OMB code.gov and OGC Intellectual Property (IP) to develop core contract language to disseminate within the Department. DHS CIO, in coordination with OCPO, OGC, and other stakeholders, ensures appropriate contract language, and a mechanism to ensure that delivered software meets contractual requirements, is in place.

DHS Component Responsibilities

This policy requires DHS Components to account for their custom-developed code, facilitate government-wide code reuse, and publish a portion of their custom-developed code as open source in accordance with OMB M-16-21 including:

• A new baseline of software development and acquisition that supports a “default to open” approach as it promotes the Department's vision of appropriate transparency.
• A posture of “default to open” requiring justification for noncompliant software, and ensuing delivery of custom-developed code, documentation, and other associated materials from the developer throughout the development process.
• A dissemination process for custom-developed code and related information that maximizes its availability to all other federal agencies, subject to limited exceptions, and that is the responsibility of the organization releasing code.
• An adherence to releasing a portion of custom-developed code as OSS through a public- facing software version controlled platform in a manner whereby any party can contribute new code, modify existing code, or make other suggestions to improve the software through the software development lifecycle.
• Contracts that must follow OMB’s three-step software analysis outlined in OMB M-16-21 and include contract requirements for OSS, open standards, and modular architecture requirements when applicable. Contracts for custom-developed code must also acquire and enforce rights sufficient to enable government-wide (or perhaps public) reuse of custom-developed code; and .
• The inclusion of a metadata file in each project’s source code repository. The metadata file will contain information about the project that can be included in the Department’s code inventory.

Note that DHS Component responsibilities will require a number of stakeholders from within the Component including the Component's OCIO, OGC, security department, project managers, and source-code developers. DHS requires that each Component identify a representative to lead the implementation of the Federal Source Code Policy and to coordinate with DHS OCIO to ensure that the Component is compliant with all inventory, open source release and inventory submission policies and requirements.

Exceptions

The exceptions provided below may be applied, in specific instances, to exempt the Department from sharing custom-developed code with other government agencies. Any exceptions used are approved, in consultation with OGC, and documented by the DHS CIO (and provided to OMB) for the purposes of ensuring effective oversight and management of information technology resources. Applicable exceptions are as follows:

• The sharing of the source code is restricted by law or regulation;
• The sharing of the source code would create an identifiable risk to the detriment of national security, confidentiality of government information, or individual privacy;
• The sharing of the source code would create an identifiable risk to the stability, security, or integrity of the Department’s systems or personnel;
• The sharing of the source code would create an identifiable risk to the DHS mission, programs, or operations; or
• The DHS CIO believes it is in the national interest to exempt sharing the source code.

The inventory process involves identifying and recording all DHS custom-developed code.

Version Control Systems

DHS Components must identify the locations of all custom-developed code. For DHS, all custom-developed code must reside in a Version Control System (VCS). A VCS is a system for managing revision control of software. VCS systems include GitHub, GitLab and Bitbucket (Git); Apache Subversion (SVN); and Concurrent Versions Systems (CVS). For DHS, VCSs may be located internally or externally to DHS and may be privately or publicly accessible.

VCSs typically package related files into units called projects or repositories. However, the semantics surrounding these terms vary depending on the VCS. In this document, we use the term project to refer to a package of related files which may or may not include custom-developed source code. In addition, we use the term source code project to refer to a package containing DHS custom-developed source code (along with related files and artifacts). Here, we note that the term 'source code project' is consistent with both Policy Directive 142-04 and the code.gov Metadata Schema.

This document also distinguishes a project from a repository by defining a repository as a VCS that may or may not include source code projects. Similarly, we define a source code repository as a VCS that contains one or more source code projects.

Identifying Custom-Developed Code

DHS Components must identify all source code projects containing DHS custom-developed code (i.e., DHS source code projects). OMB M-16-21 defines custom-developed code as:

"[C]ode that is first produced in the performance of a Federal contract or is otherwise fully funded by the Federal Government. It includes code, or segregable portions of code, for which the Government could obtain unlimited rights under Federal Acquisition Regulations (FAR) Pt. 27 and relevant agency FAR Supplements. Custom-developed code also includes code developed by agency employees as part of their official duties. For the purposes of this policy, custom-developed code may include, but is not limited to, code written for software projects, modules, plugins, scripts, middleware, and APIs; it does not, however, include code that is truly exploratory or disposable in nature, such as that written by a developer experimenting with a new language or library."

For DHS, this definition of custom-developed code is extended to include any custom-developed artifacts related to DHS custom-developed code including documentation, configuration files, and other artifacts necessary for the continued development, maintenance or execution of the custom-developed code.

Note that some projects contained in a repository may or may not include DHS custom-developed code. In some cases, projects may only include artifacts such as COTS code, open source code from other developers, and documents unrelated to DHS-specific software development efforts. Such projects should not be included in the DHS source code inventory.

Identifying Usage Type

DHS Components must designate the usage type for each source code project. The usage type for a source code project denotes the usage permissions for that project. Per the specifications as outlined in the code.gov Metadata Schema specification, there are three general usage types for a source code project: open source, government reuse (i.e., government-only), and exempted. If a source code project is intended for public open source release, the project should be defined as having a usage type of openSource. Similarly, if a source code project is intended for use only within the federal government, the repository should have a usage type of government-wide reuse.

In some cases, a source code project will not be intended for open source release or government reuse but instead will be intended for private or closed use and exempt from the "default to open" posture of DHS. Per the code.gov Metadata Schema, the usage type for exempted source code projects is based on the following exemption justifications:

• exemptByLaw: The sharing of the source code is restricted by law or regulation, including—but not limited to—patent or intellectual property law, the Export Asset Regulations, the International Traffic in Arms Regulation, and the Federal laws and regulations governing classified information.
• exemptByNationalSecurity: The sharing of the source code would create an identifiable risk to the detriment of national security, confidentiality of government information, or individual privacy.
• exemptByNationalSecurity: The sharing of the source code would create an identifiable risk to the detriment of national security, confidentiality of government information, or individual privacy.
• exemptByAgencySystem: The sharing of the source code would create an identifiable risk to the stability, security, or integrity of the agency’s systems or personnel.
• exemptByAgencyMission: The sharing of the source code would create an identifiable risk to agency mission, programs, or operations.
• exemptByCIO: The Component's CIO believes it is in the national interest to exempt sharing the source code.
• exemptByPolicyDate: The release was created prior to the M-16-21 policy (August 8, 2016).

Determining the usage type of source code projects determines the processes applied to those projects. For example, a source code project intended as open source will undergo a process for determining the appropriate open source licensing as well as a process for analyzing the security of the custom-developed code before the project can be transitioned to a publicly-accessible repository. Likewise, a source code project designated as exempted will undergo a process for obtaining an approval for the stated exemption justification. Components should consult with their respective OCIO, OGC, security and other offices to approve usage designations for all custom-code projects. Table 2 shows source code project usage types and their corresponding requirements.

Note that all source code projects must be inventoried even if they are exempted from open source release.

Defining Metadata

DHS Components must define metadata for each of their source code projects. This metadata will be used by DHS to provide a comprehensive inventory description of all DHS custom-developed code.

The metadata required for each DHS source code project is defined by the code.gov Metadata Schema. This schema defines both mandatory and optional metadata fields for source code projects. For DHS, metadata for a source code project must contain the mandatory metadata fields as defined by the code.gov Metadata Schema. DHS also recommends that source code projects include any optional code.gov Metadata Schema metadata fields. Note that although tools exist for automatically gleaning metadata from source code repositories (e.g., LLNL Scraper), such tools typically do not acquire all of the metadata required by code.gov Metadata Schema. Regardless of the method used to acquire metadata for a source code repository, DHS Components are required to ensure that all mandatory metadata fields for a source code project are captured.

Additional Requirements

In addition to the mandatory metadata fields specified in the code.gov Metadata Schema, DHS Components must also implement additional metadata field requirements as specified in Table 3.

JSON Metadata File

code.gov requires agencies to document their source code projects inventory using a Javascript Object Notation (JSON) file. This file contains all of the metadata defined for each source code project. The following example JSON file shows DHS mandatory (in black) and optional (in purple) fields. DHS-specific guidance is denoted in green comments (note that comments are not part of the JSON specification). This example includes two source code projects: (1) an open source repository and (2) an exempted (private) repository.

{

/* This example JSON file contains two example source code projects: an open source repository and an exempted (private) repository. */

/* Version of the code.gov Metadata Schema. */

"version": "2.0.0",

/* Always use 'DHS' for agency. */

"agency": "DHS", "measurementType": {

/* DHS uses the number of source code projects for measuring inventory. */

"method": "projects" }, "releases": [ {

/*----- PROJECT 1: Open-source example. -----*/

/* Name of the repository as it appears in the repository. */

"name": "daiml",

"version": "1.0",

/* DHS Component acronym (e.g., MGMT, FEMA, etc.), office, and division. */

"organization": "MGMT, OCIO, OCTO", "description": "A platform for demonstrating AI code.", "permissions": {

/* DHS-mandatory if intended for open source. */

"licenses": [ { "URL": "https://api.github.com/repos/dhs-gov/daiml/LICENSE", "name": "MIT", } ], "usageType": "openSource" }, "tags": [ "github", "AI", "Artificial Intelligence", "ML", "Machine Learning", "Demo", "Testbed" ], "contact": { "email": "stephen.quirolgico@hq.dhs.gov", "phone": "202-123-4567", },

"status": "Development",

"vcs": "git",

"repositoryURL": "git://github.com/dhs-gov/daiml.git",

"homepageURL": "https://github.com/dhs-gov/daiml",

"languages": [ "Java", "CSS", "HTML" ],

/* For DHS, this value should be 0.0. */

"laborHours": 0.0,

"date": {

"created": "2018-03-20",

"lastModified": "2018-03-31"

}

}, {

/*----- PROJECT 2: Exempted example. -----*/

/* Name of the repository. */

"name": "qproject",

"version": "1.0",

/* DHS Component acronym (e.g., MGMT, FEMA, etc.), office, and division. */

"organization": "MGMT, OCIO, OCTO",

/* Brief description still required for exempted code. */

"description": "A quantum cryptography service.", "permissions": {

/* DHS-mandatory if intended for open source. */

"licenses": null, "usageType": "exemptByNationalSecurity",

"exemptionText": "This software is exempt as sharing of this code would create an identifiable risk to the detriment of national security."

}, "tags": [ "bitbucket", "Quantum Cryptography", "Zero Trust" ], "contact": { "email": "stephen.quirolgico@hq.dhs.gov", "phone": "202-123-4567", },

"status": "Development",

"vcs": "git",

/* For exempted projects, repositoryURL is optional. */

"repositoryURL": "git://maestro.dhs.gov/CRYPT/qproject.git",

/* For DHS, this value should be 0.0. */

"laborHours": 0.0,

"date": {

"created": "2019-03-20",

"lastModified": "2019-10-01"

}

} ] }

DHS Components must create a JSON metadata file that captures all of the Component's source code projects. To assist in the development of this JSON file, a template for DHS Components is available. DHS Components must also ensure that their JSON file complies with the code.gov Metadata Schema. To verify compliance of your JSON file with the code.gov Metadata Schema, DHS Components should test their JSON metadata file using the code.gov Metadata validator.

Pursuant to the requirement for DHS to form a new baseline of software development and acquisition that supports a “default to open” approach, the SCIP specifies an open release process that involves preparing custom-developed code for release on a publicly-accessible repository. DHS Components must identify a minimum of 20% of "new" source code projects (i.e., projects created in 2016 or later) to be released as open source. The open source release requirement does not apply to source code projects with usage type government-wide reuse or that have been exempted by the justifications described in Section 2.3.

Licensing

Before a source code project can be released as open source, the project must apply an appropriate open source license to the source code project. Selecting an appropriate open source license is dependent on many factors that often have implications with respect to intellectual property (IP) including whether the project was developed by federal employees or contractors whether the project contains licensed artifacts such as libraries or data. DHS Components are required to consult with their Component's OCIO and OGG offices to determine the appropriate licensing for a source code project and to obtain their Component's OCIO and/or OGC approval for the use of the identified open source license.

Security Assessment

Before a source code project can be released as open source, a security assessment of the project must be performed by the Component's OCIO. Such an assessment should ensure that the project does not included sensitive DHS data and that the project is free of malware, viruses or any critical vulnerabilities that may harm the security of end users or their systems. Security approval by the Component's OCIO is required before a source code project can be transitioned to a public repository.

OGC Review

Before a source code project can be released as open source, the project must be assessed by the Component's OGC. Assessments may include legal analysis of export controls, technical transfer, etc. Legal approval by the Component's OGC is required before a source code project can be transitioned to a public repository.

Project Transition

After approval by the DHS Office of General Counsel (OGC), a request should be made to the DHS Source Code Inventory Process team (scip@hq.dhs.gov) to request permission to transition code to the official DHS public repository at https://github.com/dhs-gov.

Metadata Update

After transition of a source code project to the official DHS public repository, metadata for the source code project should be updated and provided to the Component's SCIP representative to update the Component's source code inventory.

General Requirements

DHS Components are required to submit their complete JSON inventory file (following the schedule defined in Section 4.2) each time their inventory changes. If the source code inventory for a Component has not changed since their last submission, Components do not need to resubmit their inventory.

Schedule

DHS Components are required to submit their JSON inventory file on the 1st Monday of every month only if their inventory has changed since their last submission.

JSON Verification

Before a JSON inventory file can be submitted to the DHS OCIO, the Component must first ensure that JSON file complies with the code.gov Metadata Schema. Components should use the the code.json Validator Tool.

JSON Submission

After validating a JSON inventory file, the JSON inventory file may be sent as an email attachment to the DHS Source Code Inventory Process team.

Components should consult with their associated SCIP representatives for information about their specific processes, policies and requirements.