Data Governance
How to make government interoperable, one step at a time
Data governance is the practice of organizing and implementing policies,
procedures and standards that maximizes data access and interoperability
for the business mission.
The method discussed here offers a practical and simple process for integrating
very complex data systems. Integrating siloed data automatically solves
inefficient business processes throughout the organization.
Background
There are thousands of data fields and tables in every organization that
need to be standardized and analyzed for optimal reuse. Left unanalyzed,
data designs create undue complexity and siloed data that does not interoperate
with other systems. This lack of interoperability is manifested on the
business side as inefficient business processes.
To address the time consuming data analysis process, the creation of
a Data Governance Council and integration prioritization method is described
here that reduces the task to a quick review of a list of the most important
fields and tables.
Super Connectors
Tables and fields that have the most intradepartmental and nationwide
connectivity are called super-connector tables and super-connector fields.
Super-connectors cut across agencies in state and federal government.
They can be used to create an intelligent neural network that allows government’s
hand to know what its other hand is doing
An example of a super connector is the case number for a criminal defendant
as it is used by police, district attorneys and judges. If multiple identification
numbers are used to identify the case, by the time it reaches the courts,
judges may become confused as to which identifier they should use. (Jailhouse
Talk - Case Files - Integration Initiatives - CIO Magazine Mar 1, 2003
http://www.cio.com/article/31738/Integration_Initiative_for_Maricopa_County_Law_Enforcement)
If the case number super connector becomes standardized across government,
greater business efficiency will result.
Another example: Police departments have a database of recent crimes.
The parole board has a database of parolees. A crime category field exists
in both systems. Normalized warehoused data using super connectors unlocks
a new benefit where the police can now download data on all recently released
parolees that have previously committed the matching crime and have an
instant list of suspects.
Another super connector is many states' professional license number used
by boards and bureaus and is used by DOJ. Corporation number is used by
the state licensing agencies, secretaries of state and many other state
and federal organizations.
Summary Overview
The governance process would review all new software development to see
if it could be shared enterprise-wide. Whenever new systems are designed,
each field and table would be checked to see if it is on the super-connector
list. If yes, the field or table would be standardized and the Data Governance
Council would analyze interoperability opportunities within state and
federal government. Redundant tables would be replaced by a single centralized
table that would contain the most accurate, up-to-the-moment information.
This method allows government to reduce the crippling complexity in its
data systems without the large costs of replacing them.
It also creates a sustained process that works to keep IT continually
aligned with the business mission.
Data Governance Process
The data governance process is intended to build collaboration into government
organizational structures and expand it from data integration to business
process integration.
Data Governance Council
The Data Governance Council would be comprised of three groups: Business
clients, developer and DBA community, and a data architect.
The purpose of the council is to (1) bring business people into enterprise
data planning as full partners with IT professionals and (2) get the maximum
number of minds, including business analysts and computer programmers,
to collectively build an integrated data vision.
Business clients - Volunteers from any part of the business side.
In general, business clients describe what they want from the data.
- Identify data needs so that they can be documented in a centralized
list and prioritized
- Review reports of new fields and tables. Provide feedback on usability
of new fields and tables
- Collectively discuss and prioritize interoperability opportunities
in the data governance forum
- Identify new sources of data
- Assist in creating data standards and a business glossary (data dictionary)
including harmonizing and standardizing data names and definitions
- Make “survivorship” decisions, e.g., if more than one record
for the same person exists, which one should be deleted
- Recommend data policy including improvements to data governance process.
- Contribute to the organization's data strategy and statewide data
vision
- Collaborate with outside communities of interest
Developer and DBA community – Volunteers from any part of
the IT side. The developer and DBA community will collectively design
data so that business clients have the data they need, when they need
it. Their primary methodology is strict normalization of all data files,
which automatically reveals integration opportunities in the most efficient
manner.
- Identify new fields and tables and forward them to data architect
so that the data architect can add it to the organization's data model
- Identify new sources of data
- Build new fields and tables in compliance with data standards
- Review reports of new fields and tables. Provide feedback on usability
of new fields and tables
- Collectively discuss and prioritize interoperability opportunities
in the data governance forum
- Assist in creating data standards and a data dictionary including
harmonizing and standardizing data names and definitions
- Recommend data policy including improvements to data governance process.
Contribute to the organization's data strategy and statewide data vision
- Collaborate with outside communities of interest
Data architect – The data architect primarily maintains
the enterprise-wide data design so that the business and IT communities
can collaboratively plan enterprise integration.
- Advise the organization on enterprise integration issues and recommend
data policy
- Create roadmap to enterprise integration and use it to coordinate
enterprise integration in collaboration with internal and external business
and IT communities
- Review all data design plans at the conception stage and look for
hidden integration opportunities or problems
- Harmonize and standardize data in collaboration with development and
DBA community
- Maintain a centralized list of prioritized table and field super-connectors
with a table of connection instances and opportunities. Communicate
super-connector standards to new systems designers
- Maintain meta data repository
- Integrate data governance into all system design, including that performed
by contractors and vendors. Follow up and verify during system construction
stage and after system completion
- Display a list of proposed data designs in a centralized location
so that all stakeholders can analyze and leverage integration opportunities
- Maintain a calendar
of scheduled system builds and system changes that identify potential
integration opportunities
- Integrate data governance into government policy and processes such
as FSR, RFPs, SDLCs, PMO methodology, etc.
- Collaborate with outside communities of interest. Research trends
and identify opportunities with other data governance organizations
- Integrate the Federal Data Reference Model with the organization's
data governance processes
- Maintain online data governance forum
- Measure performance – Maintain a transparent database of (a)
successful and unsuccessful integration projects (b) current data problems
(c) business customer requests
- Create data governance quality management process
- Obtain a department-wide data modeling tool and maintain department-wide
data model
- Contribute to a statewide data vision
- Build the DNA of continual process improvement into the structure
of every data governance process
Data Governance Workflow
Workflow Summary
Data architect is notified when new data designs are planned and then
publishes the changes for Data Governance Council analysis.
Data Governance Council reviews the data against a checklist,
primarily focused on integration opportunities revealed by the presence
of super-connectors.
If super-connectors exist, then they are evaluated for (1) building connections
throughout the organization and (2) eliminating redundant data and processes.
Detailed Workflow Description
Data design change planning is captured and published by data architect.
New design capture areas include the following:
New data design discovery points
- IT Governance
- Project conception
- PMO (Project management office)
- SDLC (System development life cycle)
- FSR
- SPR
- PIER
- RFP
- Change Management Board
- PIT (Process Improvement Team)
- Strategic plan
- Executive strategic discussions
- BCP
- ITPP - Information Technology Procurement Plan
- PSP - Proposal Solicitation Package
- Table and field creation process (DBA, programmer, etc.)
- Database of business-side data related requests
- Informal business projects, such as potentially sharable spread sheet
data
- Computer programmers and business clients
Whenever a new field or table is created, it is analyzed by the Data
Governance Council. Analysis consists of reviewing the field or table
against a checklist.
The checklist is a comprehensive list of opportunities that become available
during changes in business data or software.
Link to Checklist
for each new business application, table or field
Data governors use super connectors to discover common or similar activities
proposed or underway, identifying opportunities to leverage proposed activities
across the state.
If a super-connector match is found on two systems, then an evaluation
is made to see if any of the opportunities in the checklist above are
feasible. If the object is a table, an evaluation can be made to see if
the table is redundant and can be combined, thereby saving errors in data
entry and eliminating redundant programming code to maintain the table.
Major new project suggestions are submitted to IT Governance, who will
prioritize them.
Experience has shown that the Data Governance Council should not make
project design wait until they have approved it. That will turn the Council
into a bottleneck. The Council should leave designers free to do what
they want, but insist that they are notified immediately even when only
a new table is designed or updated so that the Council can study it the
same day that it was conceived. A worst-case scenario would only require
the designer to backtrack a few days work if the Council found problems.
This table analysis role would save the most money because it would prevent
expensive problems from being permanently locked into systems until they
were replaced, perhaps 10 or even 30 years later.
The Data Governance Council does not have regular physical meetings but
instead reviews data online as it is designed and presented to them on
the Data Governance forum.
Workflow examples
Example 1. A vendor is hired to build a new business application. A simple
review is made to see if any of the super-connector tables could be used.
Two tables are identified and the new system designed to eliminate two
of its own tables and use the two super-connector tables, thereby avoiding
continual redundant updates of the tables throughout the life of the system.
Data for the system is more accurate and up-to-date. As new features are
added over the years, programming code is cleaner and more error-free.
Example 2. A web programmer creates a daily report of the number of new
licenses and a total license count. The programmer asks the business side
of the Data Governance Council: “Can you use this data?” They
say “yes” and it becomes one of the daily statistics displayed
on the executive dashboard.
Data Governance Requirements
- Produce greater organizational, statewide and nationwide interoperability
- Capture data design early and correct errors before bad design becomes
integrated into the organization
- Encourage maximum number of IT and business analysts to identify and
integrate super connectors and contribute to a future statewide data
integration vision by ensuring that data decision making is convenient
- Data governance system must be sustainable and manageable
- Make stakeholder input convenient
- Data governance must have measurable benefits and a system for performance
measurement
- Timeliness – the system must respond to information requests
quickly and post its information in a timely manner
- Data governance information and strategy must be transparent to all,
including PMO, Exec, business managers, business process improvement
unit and IT Governance
- The process must be flexible and embody methods for continual improvement
Implementation Plan
Stepwise and prototyped approach:
Stage One
A quick stage I with a short time line and without lengthy discussions:
- The opportunities checklist
for new data is given to programmers and PMO for volunteer use
- PMO and DBAs simply email data architect any planned schema updates.
Reasoning: Project development is currently going on in many areas without
the benefit of data governance, which could result in permanent stove-piping
of data. Opportunities for these current projects must be caught early.
Stage Two
- IT supervisors send notification of any new data design plans to data
architect
- Data architect requests stakeholder volunteers to join data governance:
managers, programmers, DBAs and business clients including boards and
bureaus. Membership can be significantly unrestricted because anyone
can contribute opportunities to standardize and integrate business processes.
- Stakeholders review data governance process
- Adjustments to data governance process are made based on stakeholder
feedback
- Test data governance process for one unit for one area. Begin with
simple goals, e.g., work with only one group and top five or ten super-connectors
- Adjustments to data governance process are made based on stakeholder
feedback
- Test period for all units
- Adjustments to data governance process are made based on stakeholder
feedback
- Implement with continual process improvement in place
Stage Three
Expand data governance to all parent and child organizations
Data Governance Benefits
Each time there is a single integration improvement, it removes roadblocks
to the organization's mission. Data silos become accessible, clients'
problems are reduced, maintenance problems are reduced and connectivity
opportunities open up across departments. This incremental method is also
the least expensive approach.
- Greater department-wide and statewide interoperability
- Citizens receive better service from integrated government business
processes. Data will be more accurate, complete, and timely. Working
with government will be more convenient, for example, when citizens
only need to go to a single government agency to update their address
instead of multiple government agencies
- Faster identification and implementation of solutions. Data governance
methodically discovers the gaps in how IT services business and shortens
the time from problem discovery to solution.
- Reduction of data duplication
- More accurate, consistent, complete, accessible and up-to-date data
- Fraud detection is facilitated because all data field names across
the department and state are standardized
- Placing all data related requests in one place allows patterns to
be identified
- Clear documentation of the lack of integration may provide business
managers with better new project proposals
- Brings the business side into the IT improvement process. Data governance
shows the business side how to find their voice in collaborative problem
solving.
- Ease of business process refinement due to standardization of data
components
- Opportunities for harmonizing and standardizing business terms because
stakeholders are brought together in a collective review process. For
example, if identical meaning terms were "cost allocation"
and "distributed cost", stakeholders could agree to standardize
on one of the terms and remove the other from business documents such
as contracts and agreements
- Business Intelligence. Data warehouse creation simplified through
standardization of business data
- Better business side control over data, privacy and project development
- Better programming code due to correctly organized data
- Improved business decisions due to accurate data from the recognized
source of record
- Increased user business side trust in data stored within the organization's
databases
- Helps meet the enterprise’s business goals including adaptation
to changing regulatory and other environments
- Agility in responding to new opportunities
- Stop business system decay. Keeps all systems tuned to organization’s
mission and to each other so that no new system rewrites are ever necessary.
Whenever there there is change, there is an opportunity to bring the
organization’s entire data one step closer to optimal organization
(third normal form) through the data governance process. Redundant table
removal, SOA opportunities coming into focus, data harmonization and removing
business rules from programming code and placing them into tables are
some of the opportunities that change brings. Once business rules are
out of programming code and in tables, they are de-siloed and available
to be shared enterprise-wide. Legacy systems should be included because
even when they are replaced, conversion will be far easier when their
files are normalized. Also, maintenance headaches from these systems will
be greatly reduced.
Data governance will stimulate software development to move towards alignment
to government's mission where it can deliver new features to consumers
and solve problems in a more efficient way.
Super Connector Field Candidates
Corporation Number
License type and license number
SSN
Address (including apartment number)
Criminal case number
Civil case number
Agency code
Super Connector Table Candidates
Chart of accounts
Licensee table
License type table
Scope
All business data is planned to be integrated. Every new data design
will be posted to the Data Governance Council forum for analysis.
Government-wide Data Standards
NIEM facilitates data exchanges between government agencies. http://www.niem.gov/.
It develops standards, a common lexicon and an on-line repository of information
exchange package documents to support information sharing. Where practical
and appropriate, it may be beneficial to drive some aspects of NIEM national
interchange standards inwards towards the organization's core so that
all new field names and formats are standardized across the organization
using NIEM standards.
NIEM should be everyone's format of choice when data leaves the organization's
borders.
Deliverables
- Data dictionary – both (1) business terms and their meanings
and (2) technical description, format and naming standards of data fields
- Data modeling tool
- Current data model
- Daily updated report of new fields and tables for business and developer
community to review
- Data Governance Council membership list including their areas of expertise
- Wiki to contain collectively designed data models and communicate
data standards
- Forum to discuss data governance issues
- Prioritized list of data integration opportunities
- Super-connector database documenting current statewide connectivity,
assessable by all government
- Roadmap to enterprise integration – updated yearly, assessable
by all government; future data model
- Performance measurement reports, assessable by all government
- Centralized, prioritized list of business-side data needs with a convenient
method for the business side to enter their business data needs
- Meta data repository
- Calendar of scheduled system builds and system changes that identify
potential integration opportunities
- Table of federal, state and departmental regulatory mandates or voluntary
guidelines that reviewers check data against.
Scalability
Data governance is scalable.
In its simplest form, data governance involves PMO/DBAs emailing the
data architect new data plans and schemas and the data architect posting
them for a voluntary Data Governance Council to review.
Comprehensive data governance can be achieved by the addition of mandated
Data Governance Council membership from each business and IT business
process segment. Each business segment would possess (a) a data modeler
from the IT side that understands the logical configuration of the data
and (b) a business analyst that understands the segment’s business
rules and needs.
Vision
It is not feasible to repair all current systems at once. The Data Governance
Council will primarily check new data design before systems are built.
It will post important old system design projects onto a prioritized candidate-for-repair
database. This entry will be “waiting” on hold for any modifications
that might occur in the future, where budget and time allow for repair.
The vision is to eventually achieve complete enterprise interoperability
by implementing integration in a stepwise manner when systems are replaced
or maintained. Data integration will reveal, and facilitate business process
integration opportunities.
Future discussions will focus on placing data governance within IT Governance
in order to gain administrative efficiencies.
In addition, a formal department-wide policy could be considered that
would integrate into data governance, non IT, siloed computer systems
including spreadsheet applications if there is interoperability value
in them.
Expanding Data Governance Method Statewide and Nationally
To expand this method statewide, one of the few additions that would
have to be made is the creation of a centralized repository on OCIO’s
website containing super-connector information: (1) super-connector name,
definition, documentation and standards (2) subject matter expert on the
business side; someone that knows the business rules and (3) a data modeler
that understands how the data is logically connected and how to retrieve
it.
A centralized discussion forum and wiki where the statewide data plan
is collectively modeled would need to be created.
A common email name for all agency data governance councils would be
useful for cross-agency collaboration, e.g., enterprise_integration@whitehouse.gov,
enterprise_integration@omb.gov, enterprise_integration@ dmv.ca.gov, etc.
It is recommended that a centralized service desk be created to assist
all of a state's agencies in connecting data between internal and external
departments. The main goal should be to make data exchange simple and
secure. It could start out as elementary as a web page containing web
services/SOA standards. A component of the service desk could direct clients
to experts who offer SOA, security, web services and other technical help
in establishing data exchanges. This would speed up connectivity implementation
first identified by data councils in each organization.
When a government client needs to search by a super connector, they would
have the OCIO’s statewide repository of which entities have the standardized
field and how to access it.
Benefits for statewide implementation:
- For all government entities, it prioritizes which fields to standardize
and provides instruction on how to standardize them
- Also for state entities, it formally designs an enterprise architect
into additional critical decision areas where they can guide business
process development towards greater interoperability
- For OCIO, it prioritizes which areas to focus on BI
- For the state, it builds interoperability statewide in the most critical
areas
- It is inexpensive and quick to implement.
The objective is to ensure that all government organizations are working
towards integration internally and externally with Data Governance Councils
as strategic planners.
Conclusion
Data governance is a powerful tool for visualizing potential enterprise-wide
inter-operational connections hidden within the complexity of seemingly
unrelated and vast government processes. The Data Governance Council should
be created so that it can use its methodology to create neural pathways
that allow an organization to know itself and make its components self-aware
in order to act as one.
Data governance PowerPoint presentation
|
