Data cleansing: The key to eliminating data siloes for good

It’s the age of big data, which means most organisations are stocking up on massive amounts of it. However, as analyst firm Forrester found, a shocking 88% of the data that’s collected is never analysed. A major contributor to this trend is siloed data — where one department ‘owns’ certain information and keeps it isolated from the rest of the organisation. Data silos can cause an organisation to be inefficient and ineffective because team members aren’t seeing the full picture of their data. What can CIOs do about this?

The great data divide

The proliferation of new technologies has led to a business environment that is truly fragmented. Each department has their own technology to complete their tasks, but rarely do these systems talk to one another.

They may be connected through a series of file exports and imports, but they aren’t working from the same pool of information. Instead, business units are hoarding their data and putting roadblocks in the way of true data transparency.

Data silos can be horizontal or vertical across an organisation. There could be barriers between individual units; for example, the sales manager and marketing manager not sharing data with each other. Or, senior leadership could be isolated from lower management levels, whereby data from lower management levels is not shared with the CEO or CFO.

Impact on IT

The current model of collecting data into different systems is not scaling to meet business needs. The growing complexity of frontline business applications is putting a massive burden on IT, as it’s your team who ends up being tasked with maintaining siloed systems, managing the growing infrastructure of data, and putting that data to use. Even if legacy tools were updated to handle data more intelligently, IT could not afford to keep multiple copies of the same data.

Your organisation isn’t going to stop doing transactional things and collecting information into individual systems, so how can you support that top layer of delivering real information and create a common data source? The answer is data cleansing.

The Standards Australia story

One company that is currently undergoing a data cleansing project is Standards Australia, the nation’s peak body for standards. Speaking at the CIO Summit in Sydney last year, IT leader Neville Turbit told of the challenges the company experienced in their first digital transformation project, which involved transferring what used to be paper standards to pdf versions. The siloed nature of the 100-year old organisation meant that there were multiple copies of one standards document, which made it difficult to know who had the latest version.

Now, Standards Australia is planning another major digital transformation as part of the data cleansing process. Over the next 12 to 18 months, the IT team is working to consolidate all the Australian and global standards into a single repository. As Turbit said, “This involves a lot of data cleansing. A lot of standards go back 100 years – we need to weed them out and make sure we’ve got the latest versions. Contextualising search is the main driver – we need to make the process of searching for standards more efficient for users. This means implementing workflows, improving the use of collaboration tools across the organisation, and being able to curate the standards collection more valuably.”

According to Turbit, this project is about thinking about the future: We need to think about how will standards be delivered in 5 to 10 years’ time. Perhaps they’ll even be delivered through smart watches.” This means engineering the processes towards cloud and mobility to prepare for these possibilities. Our current systems and processes are slow and costly, and they can’t sustain that sort of thing.”

Breaking down the barriers

Data silos create natural barriers where they otherwise might not have existed. They force people and data to stay in their lanes and damages collaboration, clouding information clarity and presenting only a small part of the picture.

The best teams know when to focus on their own priorities, but also when to balance that with other teams and work together to share data analysis and information. They know that data should be viewed as an enterprise asset rather than a departmental asset. After all, you’re all playing for the same team, and fighting for the same championship.

Are you a CIO leading a digital transformation project in your own company? Click here to download our how-to guide for advice on how to get the rest of the business on board.

Tags big data, big data analytics, digital data, Digital Transformation


Align your business strategies with the business goals