Center for Government Interoperability - Gov Ideas
Home
About
Contact
Join Us
App
Marketplace
Free Meeting Place for
Government App Sharing
Collective Prototyping for
Government Software Development
Multiple-Discipline
Solutions
Citizen
Engagement
App
FAQ

Blueprint For Better Government

Center for Government Interoperability
http://gov-ideas.com/

Center for Government Interoperability

Data Governance

How to make government interoperable, one step at a time

Data governance is the practice of organizing and implementing policies, procedures and standards that maximizes data access and interoperability for the business mission.

The method discussed here offers a practical and simple process for integrating very complex data systems. Integrating siloed data automatically solves inefficient business processes throughout the organization.

Background

There are thousands of data fields and tables in every organization that need to be standardized and analyzed for optimal reuse. Left unanalyzed, data designs create undue complexity and siloed data that does not interoperate with other systems. This lack of interoperability is manifested on the business side as inefficient business processes.

To address the time consuming data analysis process, the creation of a Data Governance Council and integration prioritization method is described here that reduces the task to a quick review of a list of the most important fields and tables.

Super Connectors

Tables and fields that have the most intradepartmental and nationwide connectivity are called super-connector tables and super-connector fields.

Super-connectors cut across agencies in state and federal government. They can be used to create an intelligent neural network that allows government’s hand to know what its other hand is doing

An example of a super connector is the case number for a criminal defendant as it is used by police, district attorneys and judges. If multiple identification numbers are used to identify the case, by the time it reaches the courts, judges may become confused as to which identifier they should use. (Jailhouse Talk - Case Files - Integration Initiatives - CIO Magazine Mar 1, 2003 http://www.cio.com/article/31738/Integration_Initiative_for_Maricopa_County_Law_Enforcement) If the case number super connector becomes standardized across government, greater business efficiency will result.

Another example: Police departments have a database of recent crimes. The parole board has a database of parolees. A crime category field exists in both systems. Normalized warehoused data using super connectors unlocks a new benefit where the police can now download data on all recently released parolees that have previously committed the matching crime and have an instant list of suspects.

Another super connector is many states' professional license number used by boards and bureaus and is used by DOJ. Corporation number is used by the state licensing agencies, secretaries of state and many other state and federal organizations.

Summary Overview

The governance process would review all new software development to see if it could be shared enterprise-wide. Whenever new systems are designed, each field and table would be checked to see if it is on the super-connector list. If yes, the field or table would be standardized and the Data Governance Council would analyze interoperability opportunities within state and federal government. Redundant tables would be replaced by a single centralized table that would contain the most accurate, up-to-the-moment information.

This method allows government to reduce the crippling complexity in its data systems without the large costs of replacing them.

It also creates a sustained process that works to keep IT continually aligned with the business mission.

Data Governance Process

The data governance process is intended to build collaboration into government organizational structures and expand it from data integration to business process integration.

Data Governance Council

The Data Governance Council would be comprised of three groups: Business clients, developer and DBA community, and a data architect.

The purpose of the council is to (1) bring business people into enterprise data planning as full partners with IT professionals and (2) get the maximum number of minds, including business analysts and computer programmers, to collectively build an integrated data vision.

Business clients - Volunteers from any part of the business side. In general, business clients describe what they want from the data.

  • Identify data needs so that they can be documented in a centralized list and prioritized
  • Review reports of new fields and tables. Provide feedback on usability of new fields and tables
  • Collectively discuss and prioritize interoperability opportunities in the data governance forum
  • Identify new sources of data
  • Assist in creating data standards and a business glossary (data dictionary) including harmonizing and standardizing data names and definitions
  • Make “survivorship” decisions, e.g., if more than one record for the same person exists, which one should be deleted
  • Recommend data policy including improvements to data governance process.
  • Contribute to the organization's data strategy and statewide data vision
  • Collaborate with outside communities of interest

Developer and DBA community – Volunteers from any part of the IT side. The developer and DBA community will collectively design data so that business clients have the data they need, when they need it. Their primary methodology is strict normalization of all data files, which automatically reveals integration opportunities in the most efficient manner.

  • Identify new fields and tables and forward them to data architect so that the data architect can add it to the organization's data model
  • Identify new sources of data
  • Build new fields and tables in compliance with data standards
  • Review reports of new fields and tables. Provide feedback on usability of new fields and tables
  • Collectively discuss and prioritize interoperability opportunities in the data governance forum
  • Assist in creating data standards and a data dictionary including harmonizing and standardizing data names and definitions
  • Recommend data policy including improvements to data governance process. Contribute to the organization's data strategy and statewide data vision
  • Collaborate with outside communities of interest

Data architect – The data architect primarily maintains the enterprise-wide data design so that the business and IT communities can collaboratively plan enterprise integration.

  • Advise the organization on enterprise integration issues and recommend data policy
  • Create roadmap to enterprise integration and use it to coordinate enterprise integration in collaboration with internal and external business and IT communities
  • Review all data design plans at the conception stage and look for hidden integration opportunities or problems
  • Harmonize and standardize data in collaboration with development and DBA community
  • Maintain a centralized list of prioritized table and field super-connectors with a table of connection instances and opportunities. Communicate super-connector standards to new systems designers
  • Maintain meta data repository
  • Integrate data governance into all system design, including that performed by contractors and vendors. Follow up and verify during system construction stage and after system completion
  • Display a list of proposed data designs in a centralized location so that all stakeholders can analyze and leverage integration opportunities
  • Maintain a calendar of scheduled system builds and system changes that identify potential integration opportunities
  • Integrate data governance into government policy and processes such as FSR, RFPs, SDLCs, PMO methodology, etc.
  • Collaborate with outside communities of interest. Research trends and identify opportunities with other data governance organizations
  • Integrate the Federal Data Reference Model with the organization's data governance processes
  • Maintain online data governance forum
  • Measure performance – Maintain a transparent database of (a) successful and unsuccessful integration projects (b) current data problems (c) business customer requests
  • Create data governance quality management process
  • Obtain a department-wide data modeling tool and maintain department-wide data model
  • Contribute to a statewide data vision
  • Build the DNA of continual process improvement into the structure of every data governance process

Data Governance Workflow

Workflow Summary

Data architect is notified when new data designs are planned and then publishes the changes for Data Governance Council analysis.

Data Governance Council reviews the data against a checklist, primarily focused on integration opportunities revealed by the presence of super-connectors.

If super-connectors exist, then they are evaluated for (1) building connections throughout the organization and (2) eliminating redundant data and processes.

Detailed Workflow Description

Data design change planning is captured and published by data architect. New design capture areas include the following:

New data design discovery points

  • IT Governance
  • Project conception
  • PMO (Project management office)
  • SDLC (System development life cycle)
  • FSR
  • SPR
  • PIER
  • RFP
  • Change Management Board
  • PIT (Process Improvement Team)
  • Strategic plan
  • Executive strategic discussions
  • BCP
  • ITPP - Information Technology Procurement Plan
  • PSP - Proposal Solicitation Package
  • Table and field creation process (DBA, programmer, etc.)
  • Database of business-side data related requests
  • Informal business projects, such as potentially sharable spread sheet data
  • Computer programmers and business clients

Whenever a new field or table is created, it is analyzed by the Data Governance Council. Analysis consists of reviewing the field or table against a checklist. The checklist is a comprehensive list of opportunities that become available during changes in business data or software.

Link to Checklist for each new business application, table or field

Data governors use super connectors to discover common or similar activities proposed or underway, identifying opportunities to leverage proposed activities across the state.

If a super-connector match is found on two systems, then an evaluation is made to see if any of the opportunities in the checklist above are feasible. If the object is a table, an evaluation can be made to see if the table is redundant and can be combined, thereby saving errors in data entry and eliminating redundant programming code to maintain the table.

Major new project suggestions are submitted to IT Governance, who will prioritize them.

Experience has shown that the Data Governance Council should not make project design wait until they have approved it. That will turn the Council into a bottleneck. The Council should leave designers free to do what they want, but insist that they are notified immediately even when only a new table is designed or updated so that the Council can study it the same day that it was conceived. A worst-case scenario would only require the designer to backtrack a few days work if the Council found problems. This table analysis role would save the most money because it would prevent expensive problems from being permanently locked into systems until they were replaced, perhaps 10 or even 30 years later.

The Data Governance Council does not have regular physical meetings but instead reviews data online as it is designed and presented to them on the Data Governance forum.

Workflow examples

Example 1. A vendor is hired to build a new business application. A simple review is made to see if any of the super-connector tables could be used. Two tables are identified and the new system designed to eliminate two of its own tables and use the two super-connector tables, thereby avoiding continual redundant updates of the tables throughout the life of the system. Data for the system is more accurate and up-to-date. As new features are added over the years, programming code is cleaner and more error-free.

Example 2. A web programmer creates a daily report of the number of new licenses and a total license count. The programmer asks the business side of the Data Governance Council: “Can you use this data?” They say “yes” and it becomes one of the daily statistics displayed on the executive dashboard.

Data Governance Requirements

  • Produce greater organizational, statewide and nationwide interoperability
  • Capture data design early and correct errors before bad design becomes integrated into the organization
  • Encourage maximum number of IT and business analysts to identify and integrate super connectors and contribute to a future statewide data integration vision by ensuring that data decision making is convenient
  • Data governance system must be sustainable and manageable
  • Make stakeholder input convenient
  • Data governance must have measurable benefits and a system for performance measurement
  • Timeliness – the system must respond to information requests quickly and post its information in a timely manner
  • Data governance information and strategy must be transparent to all, including PMO, Exec, business managers, business process improvement unit and IT Governance
  • The process must be flexible and embody methods for continual improvement

Implementation Plan

Stepwise and prototyped approach:

Stage One

A quick stage I with a short time line and without lengthy discussions:

  1. The opportunities checklist for new data is given to programmers and PMO for volunteer use
  2. PMO and DBAs simply email data architect any planned schema updates.

Reasoning: Project development is currently going on in many areas without the benefit of data governance, which could result in permanent stove-piping of data. Opportunities for these current projects must be caught early.

Stage Two

  • IT supervisors send notification of any new data design plans to data architect
  • Data architect requests stakeholder volunteers to join data governance: managers, programmers, DBAs and business clients including boards and bureaus. Membership can be significantly unrestricted because anyone can contribute opportunities to standardize and integrate business processes.
  • Stakeholders review data governance process
  • Adjustments to data governance process are made based on stakeholder feedback
  • Test data governance process for one unit for one area. Begin with simple goals, e.g., work with only one group and top five or ten super-connectors
  • Adjustments to data governance process are made based on stakeholder feedback
  • Test period for all units
  • Adjustments to data governance process are made based on stakeholder feedback
  • Implement with continual process improvement in place

Stage Three

Expand data governance to all parent and child organizations

Data Governance Benefits

Each time there is a single integration improvement, it removes roadblocks to the organization's mission. Data silos become accessible, clients' problems are reduced, maintenance problems are reduced and connectivity opportunities open up across departments. This incremental method is also the least expensive approach.

  • Greater department-wide and statewide interoperability
  • Citizens receive better service from integrated government business processes. Data will be more accurate, complete, and timely. Working with government will be more convenient, for example, when citizens only need to go to a single government agency to update their address instead of multiple government agencies
  • Faster identification and implementation of solutions. Data governance methodically discovers the gaps in how IT services business and shortens the time from problem discovery to solution.
  • Reduction of data duplication
  • More accurate, consistent, complete, accessible and up-to-date data
  • Fraud detection is facilitated because all data field names across the department and state are standardized
  • Placing all data related requests in one place allows patterns to be identified
  • Clear documentation of the lack of integration may provide business managers with better new project proposals
  • Brings the business side into the IT improvement process. Data governance shows the business side how to find their voice in collaborative problem solving.
  • Ease of business process refinement due to standardization of data components
  • Opportunities for harmonizing and standardizing business terms because stakeholders are brought together in a collective review process. For example, if identical meaning terms were "cost allocation" and "distributed cost", stakeholders could agree to standardize on one of the terms and remove the other from business documents such as contracts and agreements
  • Business Intelligence. Data warehouse creation simplified through standardization of business data
  • Better business side control over data, privacy and project development
  • Better programming code due to correctly organized data
  • Improved business decisions due to accurate data from the recognized source of record
  • Increased user business side trust in data stored within the organization's databases
  • Helps meet the enterprise’s business goals including adaptation to changing regulatory and other environments
  • Agility in responding to new opportunities
  • Stop business system decay. Keeps all systems tuned to organization’s mission and to each other so that no new system rewrites are ever necessary.

Whenever there there is change, there is an opportunity to bring the organization’s entire data one step closer to optimal organization (third normal form) through the data governance process. Redundant table removal, SOA opportunities coming into focus, data harmonization and removing business rules from programming code and placing them into tables are some of the opportunities that change brings. Once business rules are out of programming code and in tables, they are de-siloed and available to be shared enterprise-wide. Legacy systems should be included because even when they are replaced, conversion will be far easier when their files are normalized. Also, maintenance headaches from these systems will be greatly reduced.

Data governance will stimulate software development to move towards alignment to government's mission where it can deliver new features to consumers and solve problems in a more efficient way.

Super Connector Field Candidates

Corporation Number
License type and license number
SSN
Address (including apartment number)
Criminal case number
Civil case number
Agency code

Super Connector Table Candidates

Chart of accounts
Licensee table
License type table

Scope

All business data is planned to be integrated. Every new data design will be posted to the Data Governance Council forum for analysis.

Government-wide Data Standards

NIEM facilitates data exchanges between government agencies. http://www.niem.gov/. It develops standards, a common lexicon and an on-line repository of information exchange package documents to support information sharing. Where practical and appropriate, it may be beneficial to drive some aspects of NIEM national interchange standards inwards towards the organization's core so that all new field names and formats are standardized across the organization using NIEM standards.

NIEM should be everyone's format of choice when data leaves the organization's borders.

Deliverables

  • Data dictionary – both (1) business terms and their meanings and (2) technical description, format and naming standards of data fields
  • Data modeling tool
  • Current data model
  • Daily updated report of new fields and tables for business and developer community to review
  • Data Governance Council membership list including their areas of expertise
  • Wiki to contain collectively designed data models and communicate data standards
  • Forum to discuss data governance issues
  • Prioritized list of data integration opportunities
  • Super-connector database documenting current statewide connectivity, assessable by all government
  • Roadmap to enterprise integration – updated yearly, assessable by all government; future data model
  • Performance measurement reports, assessable by all government
  • Centralized, prioritized list of business-side data needs with a convenient method for the business side to enter their business data needs
  • Meta data repository
  • Calendar of scheduled system builds and system changes that identify potential integration opportunities
  • Table of federal, state and departmental regulatory mandates or voluntary guidelines that reviewers check data against.

Scalability

Data governance is scalable.

In its simplest form, data governance involves PMO/DBAs emailing the data architect new data plans and schemas and the data architect posting them for a voluntary Data Governance Council to review.

Comprehensive data governance can be achieved by the addition of mandated Data Governance Council membership from each business and IT business process segment. Each business segment would possess (a) a data modeler from the IT side that understands the logical configuration of the data and (b) a business analyst that understands the segment’s business rules and needs.

Vision

It is not feasible to repair all current systems at once. The Data Governance Council will primarily check new data design before systems are built. It will post important old system design projects onto a prioritized candidate-for-repair database. This entry will be “waiting” on hold for any modifications that might occur in the future, where budget and time allow for repair.

The vision is to eventually achieve complete enterprise interoperability by implementing integration in a stepwise manner when systems are replaced or maintained. Data integration will reveal, and facilitate business process integration opportunities.

Future discussions will focus on placing data governance within IT Governance in order to gain administrative efficiencies.

In addition, a formal department-wide policy could be considered that would integrate into data governance, non IT, siloed computer systems including spreadsheet applications if there is interoperability value in them.

Expanding Data Governance Method Statewide and Nationally

To expand this method statewide, one of the few additions that would have to be made is the creation of a centralized repository on OCIO’s website containing super-connector information: (1) super-connector name, definition, documentation and standards (2) subject matter expert on the business side; someone that knows the business rules and (3) a data modeler that understands how the data is logically connected and how to retrieve it.

A centralized discussion forum and wiki where the statewide data plan is collectively modeled would need to be created.

A common email name for all agency data governance councils would be useful for cross-agency collaboration, e.g., enterprise_integration@whitehouse.gov, enterprise_integration@omb.gov, enterprise_integration@ dmv.ca.gov, etc.

It is recommended that a centralized service desk be created to assist all of a state's agencies in connecting data between internal and external departments. The main goal should be to make data exchange simple and secure. It could start out as elementary as a web page containing web services/SOA standards. A component of the service desk could direct clients to experts who offer SOA, security, web services and other technical help in establishing data exchanges. This would speed up connectivity implementation first identified by data councils in each organization.

When a government client needs to search by a super connector, they would have the OCIO’s statewide repository of which entities have the standardized field and how to access it.

Benefits for statewide implementation:

  • For all government entities, it prioritizes which fields to standardize and provides instruction on how to standardize them
  • Also for state entities, it formally designs an enterprise architect into additional critical decision areas where they can guide business process development towards greater interoperability
  • For OCIO, it prioritizes which areas to focus on BI
  • For the state, it builds interoperability statewide in the most critical areas
  • It is inexpensive and quick to implement.

The objective is to ensure that all government organizations are working towards integration internally and externally with Data Governance Councils as strategic planners.

Conclusion

Data governance is a powerful tool for visualizing potential enterprise-wide inter-operational connections hidden within the complexity of seemingly unrelated and vast government processes. The Data Governance Council should be created so that it can use its methodology to create neural pathways that allow an organization to know itself and make its components self-aware in order to act as one.

Data governance PowerPoint presentation