The value and challenges of capturing data

We all know that sound business decisions require a single version of the truth. And most of us have worked on enough large-scale enterprise data warehouse projects to know that capturing data presents any number of challenges.

Application data collected from an enterprise’s various business units don’t necessarily provide sufficient context because the data are not centrally managed. We also see the challenge of shadow IT departments in separate business units that serve the unit’s needs but could be an issue at an enterprise level.

Where the IT department can implement governance to ensure consistency and standards across the enterprise, shadow IT departments have no such constraints. (The cloud exacerbates this issue because it reduces the upfront costs of shadow IT departments.) Even if an IT department succeeds in implementing a single version of the truth, overwhelming demand for data cramps IT’s ability to implement timely data feeds.

Of course, the ultimate goals are achieving a targeted, enterprise-wide growth strategy that works for all business units and ensuring efficiency across business units, Sales, Marketing, Finance, Supply Chain, and Logistics. And this all has to happen while aligning to corporate data standards, data quality and data sources.

Why compliance to data regulations is key

Another challenge is that an enterprise-wide data management strategy requires a top-down directive that includes regulatory compliance, a critical business requirement for most companies.

Unlike failure to optimize data or to create cost efficiencies, which result in self-imposed expenditures, businesses that are non-compliant with data regulations are potentially subject to externally imposed financial penalties for the corporation and fines and imprisonment for corporate officers. Thus, compliance policies need to be communicated to and adopted by all areas, including IT.

Because so many data sources are potentially major privacy concerns, data privacy challenges are especially daunting in the app economy, and the data privacy discipline is going through a digital transformation.

Smartphone apps capture data about the user’s financial transactions, healthcare, social media activity, location, and shopping habits, among other things; all of this is personal identifiable data that the app’s creator must protect. Safeguards using encryption methods can be put in place; however, collection and storage are in digital format by default.

The business value of data can be game changing

Organized data drives the strategic analysis and decision-making that today’s multi-dimensional companies need to thrive and grow. Business leaders must understand the entire portfolio of services they offer and draw the line on what’s in and what’s out.

Decisions should be made strategically, with an emphasis on market conditions, the demand for services, and the cost of business assets. The challenge, of course, is in the details—the data we need to collect, store and analyze.

In the first blog in this series, my co-author/blogger Rob Zuurdeeg and I discussed the fact that success in the app economy depends on managing the information that powers our apps. Our second blog concentrated on the information lifecycle design. Our next blog will consider the challenges outlined above and discuss best practices in data organization and knowledge management, which has become a major discipline as the way to identify, access, maintain and use information efficiently.

In the meantime, if you’re encountered challenges other than those we talk about here, we’d love to hear from you.

Darren Arcangel is a Senior Principal Services Architect at CA Technologies. He has been the…


Modern Software Factory Hub

Your source for the tips, tools and insights to power your digital transformation.
Read more >
Low-Code Development: The Latest Killer Tool in the Agile Toolkit?What Are “Irresistible” APIs and Why Does Akamai's Kirsten Hunter Love Them?Persado's Assaf Baciu Is Engineering AI to Understand How You Feel