The following three part blog provides a comprehensive description of the concepts concerning “Data Management” specifically and a “Capability” in general within the context of a framework comprised of ten (10) data management functions.  The Data Management functions defined within this blog come from the writings of The Data Management Association (DAMA – www.dama.com) and its Data Management Body of Knowledge (DMBOK v.1).  The Data Management Body of Knowledge is a respected publication which attempts to formulate the concept of managing a corporate’s data as an asset.  The DAMA DMBOK v.1 is the basis for the terminology used to define the ten (10) data management functions.

The framework developed in writing this blog is a concept introduced by Estrada Consulting, Inc. and the author of this blog.  The framework which defines the ten (10) data management functions is used throughout this blog and is intended to allow the reader to overlay each framework diagram representing the people, tools and processes necessary to enable the data management functions.

Part 1 of this three part blog will define the Data Management framework in general with the ten (10) Data Management functions and then incorporate other elements necessary to enable Data Management as a “Capability”.  The framework presented in Part 1 will be leveraged in Part 2 to depict the exclusive roles (people), tools and processes involved in Data Management functions.  Finally, Part 3 will describe in general what a minimal and optimal data management program may look like for your organization.

The sum of the three blog parts will attempt to explain how an approach that focuses on data management capability will enhance an organization’s ability to leverage data as an asset and enable its business strategy decisions.   Unlike approaches that focus on information technology to enable business strategy, data management is a more comprehensive approach that leverages business knowledge and expects business inputs to inform the data management strategy.  Data management also takes on an enterprise view of the organization and attempts to maximize business value while minimizing resource requirements.  This is attained by delivering solutions that are fit for their purpose as defined by the business and streamlining technology and processes.

 

Definitions

Data Management – A systematic approach consisting of ten (10) data management functions designed to help an organization plan for and develop strategies to manage data as an asset.

Data Management Capability – The result of successfully combining the Data Management functions with other elements inherent in any capability.  These elements include a “Purpose” statement for the data management effort, “Metrics” and a means to measure success, “People” with the necessary skills and experience, “Tools” applicable to the effort, “Processes” which are consistent yet flexible, and “information” in the form of documented business and IT policies, procedures, standards and processes.  Finally, Data Management requires corporate Infrastructure (HR, Legal, Other) to support the teams tasked to design and develop the Data Management Capability.

Data Strategy – The combination of data management capabilities defined to address a specific business strategy.  The combination of all data strategies is referred to as the Data Management Program within this blog.

Data Management Program – This can also be called a Data Management Strategy.  It is the packaged data management capabilities necessary for the organization to meet all business strategy requirements.

 

Data Management Framework

The goal for Data Management is to allow the organization to “Manage Data as an Asset”, which is still a difficult concept to understand without context.  It is easy to understand that data is valuable today for corporations to survive in the marketplace.  It is easy to understand how to manage many physical assets like your car or home.  The concept of managing data as an asset is not so clear even though we interact with it every day like we do with our car or home.  Data Management is an approach to describe how an organization does this.  Similar to the functions we perform to manage our car (secure the door, operate the pedals, define the radio stations) data management describes ten (10) functions required to manage data.

Your Company owns many assets, and like all other organizations data allows it to describe, quantify, and assess its business capabilities, making it one of the most valuable assets.  By developing the people, processes and tools necessary to establish a Data Management Program your Company will have developed data management capabilities.  It’s these data management capabilities that will enable your Company to design and implement future data strategies.  The Data Management Program for your Company must focus on establishing capabilities in each of the ten (10) data management functions shown in the following data management framework.

Figure 1: Data Management Framework

 

Data Management consists of ten (10) functions business and IT define and develop to address business strategies and manage data as an asset.  Figure 1: Data Management Framework, developed by Estrada Consulting, Inc. depicts the ten (10) data management functions.  Data Governance sits at the top of the Data Management framework and is established to help the organization make better decisions informed by its knowledge of prior projects, business and IT metadata, and Data Management activities and deliverables.

Business strategies define a set of data requirements that typically necessitate the engagement of multiple data management functions.  Depending on the business objective an organization will typically engage a number of data management functions to satisfy the data related requirements.  As an example if the business objective is to develop a mobile application to capture customer preferences for a product it sells, this will necessitate the need to engage the Data Security, Development and Operations functions.  It would not necessarily require the engagement of other data management functions such as Document and Content Management nor Metadata Management, which are necessary to solve different business objectives.  Therefore, not every organization will build a data management capability around all the data management functions.  However, most large organizations will need to build a data management capability for each of the data management functions with varying levels of maturity.

 

Data Management Functions

A brief definition of the ten (10) data management functions is given below, which will be referred to throughout this blog.

Data Governance:  Gathers business and technical information from Data Stewards, Business Analysts, Subject Matter Experts and IT Staff representing all data management function activities for the enterprise to make tactical and strategic decisions to best utilize data as an asset for the organization.  Data Governance oversees data management function activities and reports progress to the respective Governance Council.

Data Operations Management:  Develops strategies to ensure business and IT systems are performing based on business defined service level agreements for availability and uptime.  Best practices include maintaining M&O contracts and including M&O costs in future budget forecasts.

Data Architecture Management:  Defines enterprise entities and relationships from concept to physical implementation to guide the implementation of strategic data initiatives for the enterprise.

Data Development:  Develops database artifacts and user interfaces to capture and present business and IT system data.  Data Development is focused on the data captured in COTS and Custom Business Applications and produces plans to enhance data value or mitigate data errors.  Best Practices include the System Development Life Cycle methodology to application development and Service Oriented Architecture to develop reusable service utilities.

Data Security Management:  Develops and Implements data classification models and ensures data is kept in a secure manner based on its classification.  Security Management incorporates activities to secure the network, file and database environments.  Best practices for Access Control include: Federated and Centralized Access Control and RBAC. Best practices for categorization and protection include FIPS Pub 199 and NIST (800-53).

Data Quality Management:  Develops and implements quality assurance and control activities based on business defined data quality standards.  Best practices for understanding business need include: Observation Pattern, Concept of Operations, Process-Based Information.

Document and Content Management:  Develops strategies to categorize and index documents and business content made available through file management and web-based tools.  Other services important to this function include digital signature and archival.

Data Warehousing and Business Intelligence Management:  Develops strategies to integrate and prepare enterprise data for presentation in support of enterprise business information initiatives.  Common patterns used for exchanging data needed include: Message Board Integration, Publish/Subscribe Pattern, Transaction Interception, Hub and Spoke.  Patterns for structuring data for use include: data warehouse star schema, data mart, operational data store.

Metadata Management:  Develops strategies to manage data about enterprise data including but not limited to physical data definitions, business rules, data lineage and business application feature documentation.  The artifacts delivered by a successful metadata program is central to seamless communication of Business and IT knowledge to develop other Data Management function capabilities.

Master and Reference Data Management: Develops strategies to ensure the quality and integrity of the “Golden Record” from its source of truth through to the reporting environment.  Develops data lineage documentation and works with data architects to ensure models represent the business’ data definitions.  Best practices include: Master Data Management and Data Dictionaries.

 

Data Management as an Enterprise Capability

The Data Management Program for any corporation is the combination of data management capabilities, which enable business strategy, are right-sized and sustainable.

Figure 2: Data Management as an Enterprise Capability

 

In order to establish a data management capability that is sustainable for any of the ten (10) data management functions, the organization needs to ensure the full capability is defined.  Depicted in Figure 2 is a representation of the required elements of a successful and sustainable “capability.”  These elements include: Purpose, Required Results, People, Tools, Processes, Metrics, Information, and Infrastructure.

The most important items to define at the outset of any Data Management effort include the Purpose statement for the effort and the Required Results.  These items are portrayed as the foundation of the Data Management capability in Figure 2.  No enterprise capability can last if the execution of its purpose cannot deliver value.  Data Management should have one or more of the following items as its purpose:

• Ensure the business collects the data required to perform to its mission requirements.
• Ensure the data collected is secure (protected against unauthorized modification, deletion, or addition).
• Ensure the data used in the business is the correct data for the use with the appropriate meaning to those using the data.
• Prepare the organization for being a data-driven organization where decisions are enabled with the appropriate data and associated analysis to inform key decisions.

Required results are synonymous with outcomes.  Outcomes expected from having a Data Management capability include:

• Curated data (data that has been cleaned and validated for the purpose of its use) is available for key business operations and decisions.
• Data is secure and not impacted by unauthorized alteration.
• Data is managed as prescribed by the data owners and stewards.

Given that the Data Management capability has to achieve a known set of outcomes, the processes become clearer.  Below are examples of essential data management processes, which have been categorized within their respective data management function.

• Data Governance
  • Review/Approve/Monitor Data Management Projects
  • Develop the Data Governance charter
  • Maintain data governance policy, procedure, and process documentation
  • Review and Approve Business and IT Policies, Procedures and Standards
• Data Security Management
  • Categorization of data to FIPS Pub 199
  • Establish protection through implementation of NIST Controls (800-53)
  • Detect and Protect from Internal/External Data Security Threats
• Data Operations Management
  • Develop and Implement System Back Up and Recovery Policies and Procedures
  • Monitor and Performance Tune On-Premises Servers/ Configure Cloud Servers
  • Code/Application Release Management
• Data Development Management
  • Define/Develop Data
  • Define/Develop Data Extracts
  • Define/Develop Data and Coding Standards
• Document and Content Management
  • Define and Develop Document/Record Taxonomy
  • Maintain Document Library
  • Define/Apply Document Archival Policies
• Metadata Management
  • Define and Develop Metadata Taxonomy and Standards
  • Prepare Metadata for Use (Templates, Forms, Format Specifications)
  • Maintain Metadata Catalog
• Data Warehouse and Business Intelligence Management
  • Profile Data
  • Define/Develop Data, Reports, Cubes and Visualizations
  • Define/Develop ETL Processes
  • Define/Develop Data Analytics/Science Methodologies
• Data Quality Management
  • Audit Data against Documented Business Processes, Procedures and Rules
  • Audit Data against Documented IT Data Standards, Validation Rules  and Processes
  • Review and Document Business Requirements
• Master and Reference Data Management
  • Create Data Flow Diagrams
  • Document Data Lineage
  • Define and Document Enterprise Master and Reference Data
• Data Architecture Management
  • Define and Develop Enterprise System and Data Architecture Models
  • Define and Develop Data Integration Processes
  • Develop Data Categorization Standards/Methodology

Resources (people) with the appropriate skills are essential to perform in the roles necessary to execute the processes.  Examples of the data management functions and the respective roles (people) are provided below.

• Data Governance
  • Data Governance Staff
  • Data Stewards, Owners, Stakeholders
  • Subject Matter Experts
• Data Security Management
  • Chief Information Security Officer
  • Network, System and Database Administrators
  • Compliance Analyst
• Data Operations Management
  • Server Administrator
  • Help/Service Desk Staff
  • Release Manager
• Data Development Management
  • Data Developer
  • User Interface Developer
• Document and Content Management
  • Technical Writer
  • Librarian
• Metadata Management
  • Technical Writer
  • Data Owner, Steward
• Data Warehouse and Business Intelligence Management
  • Report/Visualization Developer
  • ETL Developer
  • Data Scientist
• Data Quality Management
  • Data Quality Analyst
  • Code Quality Analyst
• Master and Reference Data Management
  • Data Analyst
  • Subject Matter Expert
  • Data Owner
• Data Architecture Management
  • Data Architect
  • System Architect

When individuals are executing the processes, there is key information needed for them to be able to accurately complete the work to deliver the process results expected.  The following list of information is likely essential:

• Business Policies, Procedures and Data Standards
• IT Application Development Policies, Procedures, Standards and Templates
• IT Data Development Policies, Procedures and Standards
• IT Security Policies, Procedures and Standards
• Detailed Business Requirements
• Business System Documentation (User and System Manuals)
• Training Literature and Videos
• Document and Data Taxonomies
• Data Quality and Performance Reports

Each individual performing in a role should be able to tell if their performance is achieving the required results for their processes.  This is possible with measurement of the outcomes.   The following list of metrics is likely essential:

• Network, System, Server and Software Performance Metrics
• Service Level Agreements
• Business Data Standards
• IT Policies, Coding Standards, Naming Conventions and Templates
• Survey Results from Data Owner, Steward, and Stakeholders

Based upon the information included above, tools and technology need to be implemented to ensure the capability is efficient and effective for the planned volumes of data needed in the enterprise.  The following list of tools and technology are likely essential:

• Database Management System (DBMS)
• Metadata Management Platform
• Network Security Appliance
• Release Management Software
• Application Development Software
• Document Management Platform
• Reporting, Visualization and Dashboard Development Software
• Testing Software
• Identity and Master Data Management Software
• Data and Process Flow Modelling Software
• Data Modelling Software

Finally, Infrastructure is the last piece essential to sustaining the data management capability.  Infrastructure represents all that the capability should take for granted from the organization, such as:

• Adequate facility space
• Responsive HR practices for hiring and related functions
• Legal counsel (if required)
• Financial processing (as applicable to data management projects).

Data Management capabilities that are developed within each of the Data Management functions will better enable the organization to address changes to data strategy as a result of changes to business strategy.  The extent to which the organization has developed the functional teams, invested in and gained the experience using Data Management tools, and implemented the processes that define its data management capability will determine the organization’s ability to align its data strategies with the organization’s business strategies.