Data Management People, Tools and Processes Defined

The following narrative will focus on defining the people, tools and processes that comprise a critical part of the data management program and support the organization’s data management capability. It will utilize a common framework template show in Figure 1, below (from our previous blog in this series, Data Management Framework).

Figure 1: Data Management Framework

 

The purpose of utilizing the same diagram to depict the people, tools and processes is to allow the reader to overlay each consecutive diagram and identify how the people, tools and processes are aligned with one another and how they relate to others.  The people, tools and processes defined in each diagram are meant to be exclusive lists of each respectively.  The intent is to blog each role (person), tool and process once where its value to the organization is at its greatest.  As your Company becomes familiar with the diagrams and the people, processes and tools, its staff can utilize the diagrams to draw relationships and where people, tools and processes will need to overlap to show the configuration that is sustainable and the right fit for your Company to meet its business strategies.

 

Data Management Roles (People)

Figure 3 below depicts the roles necessary to implement a comprehensive data management program, which should include roles from all ten (10) data management functions.  The intent of this figure is not to depict each individual role that would perform tasks within each function, rather to show the exclusive roles defined by this framework and where they provide the most immediate value when developing the data management capability.  Depending on the maturity of the data management program, multiple roles depicted below can be filled by one individual or may require multiple individuals for a given role.  For example, in organizations that have a mature data management program, the role of data developer will be filled by multiple individuals, and can be further broken down into Data Developers for custom application development and Data Developers for Data Warehouse development.  In organizations where there is minimal data warehousing and business intelligence management capabilities, one data developer may develop code to support the organization’s custom data application and data warehouse initiatives.

Additionally, some roles like Data Analyst which is placed under the Master Data and Data Warehouse Management functions assumes that the data management capability for the organization is very mature among all functions.  In this scenario the responsibilities of the Data Analyst associated with data sourcing and curation are occurring within these functions at the source (Master Data Management) and along the data transformation processes (Data Warehousing) where data analysis is most important.  However, if the organization has a less mature data management program, the data analyst role will likely exist within the Data Development Management function.

Figure 3:  People involved in Data Management Functions

 

At the top of figure 3 above and within the Data Management Framework are the Chief Information Officer, Data Governance Office Staff and Project Management Office Staff.  These resources are integral to a comprehensive data management program.  The CIO is ultimately responsible for the data strategy for the organization and therefore is accountable for the development of quality data management capabilities.  The Data Governance Office and its Staff will leverage the information contained in the Business and IT policies, procedures and standards, along with other data management function artifacts to implement its strategies and achieve its goals.  It will also influence decisions related to the tools the Data Management program will utilize and processes of which the Data Management program will need to focus.  Organizations with a mature data management program and a formalized Data Governance Office will budget for staff who will oversee and run the Data Governance activities.  Data Governance Staff are responsible for managing data governance activities including developing a data governance charter, developing the data governance committees and workgroups, scheduling and documenting meetings, monitoring Data Management activities, and maintaining data governance policies, procedures, and processes.   The Project Management Office (PMO) will provide project management capabilities for data management projects and make reports available to the data governance office when required.

Using Data Security Management as an example, the roles (people) for this function are described in the following chart.  The roles include those common throughout the framework, those specific to Data Security Management and those included in the related concentric circle drawn around it and Data Operations Management.  The term “Capacity Driven” in the table below means that the role may vary in the number of people performing it depending on resource requirements at specific point in time.

Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Chief Information Security Officer	Reports to the CIO and is accountable for the security capabilities of the enterprise.	Required	IT
Network Administrator	In charge of the design and implementation of network components that build and harden Network Infrastructure.	Required	IT
Network Security Staff	Implement the security processes and procedures defined within the data security management function to ensure data is protected at the network level.  Access Level at the Network Level.	Required, Capacity Driven	IT
Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Database Administrator	Develops standards to develop data artifacts; Access Control at the database level.  Approves and migrates database changes to the various release environments.	Required	IT
System Administrator	Responsible for the installation of and updates to Business and IT Application Software; Access Control for Business Systems	Optional (May be Performed by Server Administrator Role)	IT
Compliance Analyst	Responsible for the interpretation of security regulations and analysis of technical security compliance.	Required	Business
Data Owner	Accountable for a specific set of data required to complete job functions.  Dictates data classification and Business Data Policy Decisions.	Required	Business
Data Steward	Verifies Data is Stored and Transmitted based on data classification requirements	Optional/Capacity Driven	Business
Data Governance Staff	Apply Data Governance Policies, Procedures and Processes to enable a robust, cross-functional decision making process; Define success criteria and Monitor Performance against Defined Metrics	Required, Capacity Driven	Business and IT

 

Depending on the data management needs, there will be collaboration among staff assigned to other data management functions.  This may be in direct response to a request by a data governance committee looking to implement a new initiative that requires the input from individuals representing different data management functions.  In other situations, it may be that Data Operations management needs to implement a new data encryption procedure which requires the input from Data Security Management.  The purpose of the proposed effort, if well defined, will dictate the roles, processes and tools required to be successful.  It must be stressed that any role defined within the Data Management framework must include a detailed description of its job duties, responsibilities, processes, procedures and standards for the work it is to perform to be effective; especially for any role that is capacity driven and is a candidate to be outsourced.  If the person representing the role must rely on other people within the organization to define what and/or how to perform its duties while a project is underway, it is likely the effort will experience unanticipated delays, result in a patchwork of processes and procedures that are poorly defined and documented, and an implementation that will make data quality analysis, troubleshooting and enhancements more challenging and time consuming.

The other Data Management roles depicted in Figure 3 are presented in the following table.  The roles assume the people who carry them out are specialized in fulfilling the role responsibilities.  It is expected that the people performing these roles can provide input across multiple data management functions within their expertise and experience.

Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Server Administrator	Manages server operations to ensure availability and resilience of server infrastructure	Required, Capacity Driven	IT
Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Operations Staff (Help Desk)	Supports end users to troubleshoots user function related issues	Optional, Capacity Driven	IT
Operations Staff (Service Desk)	Supports IT users to troubleshoot and mitigate infrastructure issues.	Required, Capacity Driven	IT
Data Developer	Responsible for developing data definition, manipulation  code based on IT Standards	Required, Capacity Driven	IT
User Interface Developer (Application)	Develops UI Data Validation Code to Support the Capture of Business Data	Optional (COTS, SaaS), Required (Custom), Capacity Driven	IT
User Interface Developer (Business Intelligence)	Develops Data Presentation Artifacts (Dashboards, Public Web Reports)	Optional, Capacity Driven	IT or Business (Self-Service)
Technical Writer	Develops and Maintains policies, procedures and diagrams which define cross-functional business processes and procedures related to data management activities for the organization.	Optional, Capacity Driven	Business and IT
Librarian	Categorizes Data and Defines Document (Record) and Content Taxonomies; Maintains Data Catalog/Library	Optional	IT and Business
Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Subject Matter Expert	Provides Business Knowledge input for various data management activities; Defines Business Process and Procedures	Required	Business
Business Analyst	Works within business units to gather business requirements for new change management needs, document business process flows, policies, procedures, change requests and monitors change management requests to completion.	Required, Capacity Driven	Business
Data Analyst	Acts as the intermediary between business analyst and data developer; Define Data Definitions and Investigate new Data Sources and Processes	Required, Capacity Driven	IT
Data Quality Analyst (Business)	Analyzes Presentation/Reporting Data for Quality (UAT); Provides Input to IT Data Quality Analyst	Required	Business
Data Quality Analyst (IT)	Analyzes Data Sources and Data Preparation Methods; Performs Data Quality Control Prior to Implementation; Can be performed by Data Analyst or Data Developer (Peer Review)	Optional	IT
Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
Report/Visualization Developer	Develops Reports, File Extracts and most Advanced Visualizations in Support of Operations and Analytics Teams	Required, Capacity Driven	IT or Business (Self-Service)
ETL Developer	Develops processes and procedures to extract, transform, load (ETL) and integrate raw source data from various enterprise applications for reporting purposes	Optional, Capacity Driven	IT
Data Scientist	Develops advanced statistical models to mine data and discover data associations for AI, descriptive and predictive analyses	Optional	IT and Business
Data Architect	Designs and Develops Data Architecture (OLTP, ODS, Staging, Data Warehouse) for Transactional and Reporting Data Solutions, Develops Data Architecture Models, Develops Source/Target Mapping, Designs System Data Integration Roadmap.	Required	IT
Role Name	Role Description	Required/Optional/Capacity Driven	IT versus Business Role
System Architect	Designs system architecture strategies to implement data architecture models.  This role has extensive experience implementing data patterns utilizing key business system features.	Optional	IT and/or Business
Data Stakeholder	Business Analyst and Business Unit Managers who will be impacted by changes to Data	Required (Depending on Influence can Sink Data Initiatives if concerns are perceived to be dismissed.	Business
Audit Analyst	Analyzes and Confirms Adherence to Business and IT policies, procedures and processes; May be performed by Compliance Analyst	Optional, Capacity Driven	Business

 

Data Management Tools

The figure below depicts the Data Management tools as an overlay on top of the Data Management Framework.  Similar to how the data management roles were expressed to show where they have the most responsibility or influence, the tools shown below are also placed where the ownership of the tool should exist within the Data Management framework.  For example Data Profiling Software which is shown under the Data Quality Management function is an important tool to perform Data Quality Analysis.  Although Data Profiling Software is used by roles in other data management functions, it is not central to completing its core function.  The organization should consult the Data Quality Management staff when evaluating options for Data Profiling Software as they will have the most experience developing different processes to analyze data.

Figure 4: Data Management Tools

 

The tools have been specified within the respective data management functions in Figure 4 above to show the link between the roles (people) and the tools they use.  For example the tools specified within the Data Security Management function (Network Security Appliances, Secure Transfer Software, IPS/IDS/Firewall Software) are used by the people defined for this function including but not limited to the CISO, Security Staff, Compliance Analyst and Network Administrator.

At the top of the figure are software applications which are used in part by all data management functions.  These applications are Report, Project and File Management Software; Performance Monitoring Software to gather Network Traffic, Server, Database and Business Application Statistics; IT Service Management Software to Post and Track Business Requests, and the Business and IT application software used in business and IT management operations.

Ideally, the organization should standardize on the tools the enterprise will use to perform its data management activities.    Using Data Profiling Software as the example it is likely data quality analysts, data developers and data architects will all need to leverage the features available within this tool type.  When different tools are purchased that meet the same functional needs the financial and human resource burden for the organization grows, which takes resources away from other business or IT opportunities.  The organization should look to develop and/or procure tools that enable the required functionality as defined by the cross-functional teams who will utilize it.  It is not always apparent who will need to use some data management tools.  It will be the responsibility of the data governance team who has been tasked to identify a new IT or Business system tool to bring in a representative from each data management function within the organization to gather all necessary requirements and make the most effective decision.

The list of data management tools shown in the table below and derived from Figure 4 above is provided to begin the conversation regarding the tool set required for an organization with a mature data management program.  The Tools provided are not meant to be exhaustive, but to provide examples of the types of tools that can be used within an organization developing their Data Management capability.  The high-level costs provided for the tools are estimate ranges as the costs can vary for many of the tools depending on the source; open source versus a well-established product.  Your Company should also consider the availability of training and the level to which the tool is accepted in the industry to be assured it can find the staff to use/support them.  The cost of the tool is one thing, the maintenance and training support, as well as the cost to hire staff to use it, and the potential cost of recreating the same code when the platform is no longer available, are all important factors when considering price.  The ranges below do not account for annual licensing and maintenance fees.

Tool Type	Description	People/Process	High-Level Cost Estimate (000) per year
Network Security Appliances	Network Address Translation and IP Address Management	Network Administrator, Network Security Staff/ Monitor Network Traffic	~10 – 50 Each Appliance
Secure Transfer Software	Enable HTTPS/SFTP/TLS transmission protocols with Private/Public Key Encryption	Network Administrator. Web Developer Transfer/Transmit Data	Negligible
IPS/IDS/Firewall Software	Network Intrusion Protection/Detection	Network Administrator	~500
Patch Management Software	Monitor Server, Network, and Application Patch Releases and Install	Network, Server, and System Administrators/ Maintain Infrastructure Hardware, Software Version	Negligible
Release Management Software	Migrate applications and code to new release environment	Server and System Administrator/ Release, Migrate Code, Application	< 10
Tool Type	Description	People/Process	High-Level Cost Estimate (000) per year
Server Administration Software	Provision, Migrate, Take Offline Servers	Server Administrator/Administer Servers	~100 – 200
Database/File Back-Up and Recovery Software	Back Up Application Environments and Databases	Database and System Administrator/ Back Up and Recover Software	~10 – 100
Database Management System	Used to develop data (un/structured) storage and retrieval schemas	Database Administrator, Data Developer, Data Warehouse Developer/Develop and Manage Data	~30 – 1,000
Alert and Notification Software	Generate Automated Alerts and Notifications for IT Systems	Database Administrator, Data Developer, Data Warehouse Developer/ Develop System Automated Error Handling Messaging	< 10
Development Standards and Templates	Documented Templates and Standards for generating application, report and database code	Data Developer, Report Writer/ Develop Data and Reports, Dashboards and Visualizations	< 10
Application Development software (Custom)	Application Development Toolkit and Framework for Custom application development	Data Developer, Report Writer/ Develop Data and Reports	~100 – 250
Continuous Integration/Development Software	Development Collaboration Tool for Software Integration and Versioning	Data Developer, Report Writer, Data Warehouse Developer/ Develop Data, Report, Warehouse, ODS, OLTP	~10 – 100
Tool Type	Description	People/Process	High-Level Cost Estimate (000) per year
Report/Visualization/Analytics Software	Specialized Report Applications used for Dashboards, Visualizations and Interactive Report (OLAP)	Report Writer, Data Scientist, Data Developer/ Define and Develop Reports, Visualizations and Dashboards	~10 -250
Extraction-Transform-Load (ETL) Software	Develop data integration and transformation automated processes	ETL Developer, Data Warehouse Developer, Data Developer, Data Quality Analyst/ Extract, Transform and Load Data	~100 – 1,000
Job Scheduling Software	Create automated job schedules for on-demand and time-sensitive jobs	Data Developer, Data Warehouse Developer, Database Administrator, System Administrator/ Run and Schedule Automated Jobs	Negligible – Built into many Products
Document Management Software	Categorize, Manage, Archive, Distribute Corporate Documents	Data Owners, Data Stewards, Technical Writer, Librarian/ Manage Corporate Documents and Records	~100 – 1,000
Workflow Software	Develop workflow processes with Approval Levels	Data Developers, Business Analysts, Business Unit Managers/ Develop electronic business and IT workflow processes	~10 – 250
Digital Signature Software	Enables reduction in management of paper documents requiring signed/authorized approvals	Data Developers, Business Analysts, Business Unit Managers/ Develop electronic business and IT signed processes	~50 -100
Tool Type	Description	People/Process	High-Level Cost Estimate (000) per year
Content Management Software	Enable Self-Service change management for web-based content	Data Developers, Business Analysts, Business Unit Managers/ Manage Web Content	~10 – 100
File Management Software	Enable File Categorization and Access. Manage CRUD privileges for Corporate Files	System Administrators, Business Analysts and Managers / Manage Files and Access Privileges	Built into Windows File Server, SharePoint
Archival System Software	Enables the ability to automate processes to archive data and documents based on configured retention policies	Librarian, System Administrators, Business Unit Managers, Database Administrators / Archive Corporate Documents and Data	Part of Document Management Software
Data Catalog/Metadata Repository	Used to Develop, Store, Distribute Metadata	All Data Management People / Develop and Communicate Metadata	~200 – 250
Data Profiling Software	Used to Analyze Data for Quality	Data Developer, Data Quality Analyst, Data Warehouse Developer, Data Analyst / Analyze Data	Negligible; Also Built Into DBMS
Test and Test Automation Software	Perform tests on Data and Custom UI Code. Automate Regression Test scripts	Data Developer, Data Quality Analyst, Database Administrator, ETL Developer / Test Code	~200 – 250
Business and Technical Metadata	Definition of Data from an IT and Business Perspectives to communicate data meaning and requirements	All Data Management People / Design, Develop and Implement Business and IT Architecture, Code and Application User Interfaces.	~250
Tool Type	Description	People/Process	High-Level Cost Estimate (000) per year
Business System Documentation	System and User Manuals	System Administrators, Technical Writers, Business Analysts, Data Owners, Data Stewards / Develop Training and Business and IT Procedures	~250
Identity/Master Data Management Software	Define and Develop Master (“Golden Record”) and Reference Data	Data Analyst, Technical Writer, Data Developer, Data Architect / Define and Document Master and Reference Data	~100 – 1,000
Process Modelling Software	Develop Data and Process Flow Diagrams; Data Lineage Models	Data Architect, Database Administrator, Data Developer, Data Warehouse Developer, Quality Analyst, Business Analyst, Technical Writer/ Design Data and Business Process Flows	< 10
Data Modeling Software	Develop Entity Relationship Diagrams for Data Analysis and Application, Architecture Design	Data Architect, Database Administrator, Data Developer, Data Warehouse Developer/ Design Data Models and Architecture	< 10
Business/IT Data Standards	Documentation used to develop data, application interfaces per business and IT standards. Used to perform Data Quality Analysis	Data Developer, Data Quality Analyst, Data Architect, Data Warehouse Developer, ETL Developer/ Profile Data, Data Quality Analysis, Develop Data Architecture Models	< 100

 

It is not necessary for an organization to purchase all of the tools that are described here to enable the respective data management function.  Tools, similar to the People and Processes described in this document, must be associated with a business purpose prior to consideration.  This does not mean that IT loses its authority to purchase IT specific tools, rather that all tools that consume the organization’s financial resources should be associated with a business purpose.  IT tools that monitor system performance or protect data assets at the network level fit in with business goals of ensuring the performance and security of its data assets.  One best practice your Company should enforce to become good stewards of the financial resources it invests in data management tools is to evaluate the purchase by the data governance team and require centralized authorization to ensure the purchase aligns with business strategy and is not redundant.

 

Data Management Processes

The Data Management Processes shown in Figure 5 below are depicted within the Data Management Framework and are aligned with their respective data management functions.  By overlaying the data management framework diagrams for people, tools and processes it will become easy to see how each of the elements relate to support data management capabilities.

The processes documented below are not meant to be exhaustive.  Based on the organization’s structure and scope of work, the number of processes must be tailored to the business goals and requirements.  The processes provided are high-level processes that are considered essential within the function for which they are listed.  If the organization does not have a business need to implement a specific data management function then those processes are not necessary.  Similar to planning for staff when developing the organization’s data management program, the number of processes also depends on the specialization required.  Where there is a need to break down capabilities to a fine grain level and staff become more specialized, the number of processes will increase.  Any of the processes listed that the organization deems unnecessary should be documented as exceptions by the Data Governance committee making that determination.  The agreement by business and IT to defer a process provided in the figure below can affect the effectiveness of the organization to build data strategies that align with business strategies.  If the business later determines the capability is necessary based on a new strategy or objective, business and IT will be informed by the previous decision that the effort to implement the new data requirements can’t begin until the data management function exists.  The goal is to be capable of supporting the business strategies and avoid situations where delays in delivering on the strategies derive from poor data management planning.

The Data Management processes become clearer when analyzed within the context of an agreed-upon, well-defined purpose statement.  The execution of the processes will be more efficient and effective when based on well-defined business and IT policies, procedures, processes and standards, and applied only when necessary to achieve a business need.

Figure 5: Data Management Processes

 

The processes shown in Figure 5 above represent the essential processes within their respective functions.  A process is not complete unless there are people assigned to the process, who are trained and experienced in performing the process steps, have knowledge and experience using the tools necessary to perform the process, and there is adequate documentation in the form of policies, procedures, and standards to inform the process.  The goal for the organization is to develop processes that are repeatable and performed in a consistent fashion.  Depending on the structure of the organization and the type of products and services it delivers each of these functions can have tens to hundreds of processes defined.  The first step in defining the function’s processes will be to determine the business processes that must be enabled by data and systems, and to associate the business process requirements with one or many of the data management functions.  Furthermore, depending on the reusability of the Data Management function processes, they should be packaged and provided to Business as a Data Management Service.  Examples of packaged services include data requirements analysis, data definition and modeling, and application release services.