Data Integration – How Hard Can It Be?
TL;DR – Integration challenges can be resolved through planning, communication, collaboration and an excellent API platform.
In the year 2021, it’s natural to think that understanding your most important business data is a simple process. If you’re standing on the sidelines, you would naturally assume your team is constantly collecting, integrating and analyzing data to get the full picture of how your business is performing. You could assume that your business trends are obvious, that the next steps to take are appearing before you like magic.
But looking for patterns in your data and sussing out corrective actions before you hit a major hurdle is not normal in most wealth management firms today. One of the reasons why is because it’s so hard to get a clear picture from the data that exists.
Here are a few of the likely places your firm is currently storing information:
- Your CRM
- Portfolio Accounting Software
- Phone Systems
- Documents & Storage
- Planning Software
- Tax Tools
- Marketing Suite (Website, Web Forms, Calculators, Advertising)
- Custodians
Depending on the systems used, you may be storing your data on premises (typically legacy solutions and paper documents) or in the cloud (where most modern data sits).
Of course, there’s plenty of data overlap among systems, no matter where they are located. Key fields, where data should be integrated, often wind up in silos of information where they can’t be seen, processed or used.
No one needs to be convinced that clean, actionable data is the goal of data entry (especially not your support staff who are tasked with cleaning up records for the same household, but have to be manually digested). Systems integration, to ensure that data makes it to where it needs to be, is vital for your efficiency and growth.
Growth Hurts
The amount of data you collect daily is growing. From website stats to risk tolerance questionnaires to portfolio data, the number of fields and records your team processes keeps increasing.
As a result, your mission critical data is spread out over numerous diverse systems. To get one single view of the most important data isn’t possible if you don’t bring those disparate systems into unity. Having one single view of your data is the only sane way to make business decisions. A single view is only achieved through integration.
Application integration is an easy concept to understand, but completing integrations requires heavy lifting. Here are some of the important questions you’ll need to answer:
- What will you use as your unique client identifier across all systems?
- Which specific business objects should be integrated?
- Are we creating any dependencies by integrating from one software into another?
- Do you need real time data integration or is it enough to have scheduled jobs?
- How much data is going to transfer?
- Who will internally support the data integration process?
- Have you established in house data governance?
There are many more questions to ask, but this list is a good start for your team as you begin the process.
Scope & Data Creep
Integrating your data will provide a unified view of your business, but the difficulty of integration lies in the details. The complexity of your data integration project is impacted by:
1. The number of systems you want to include:
The more systems (some firms have hundreds) and the more diverse the system platforms are (on-prem, cloud, private) not to mention the versions add complexity. Just the security issues alone can feel overwhelming.
2. The customizations made:
If you’re using standard applications on all of your platforms, it’s much easier to understand their features and limitations. However, most integration projects include at least a few customized processes, views or code. The more custom code you’ve written in the past, the more complicated it can be to maintain, upgrade or integrate.
3. The lack of consolidated approach:
Many softwares that integrate with one another, can create a web of point-to-point custom scripts. One of the keys to making your integration project successful is to synchronize and automate across all of your platforms in a way that keeps dependencies from forming. Figuring out how data should flow through your applications can lead to fewer errors from manual data entry or errors from inconsistent data.
Solve Your Sorrows
Mapping your data correctly, even when it’s highly customized and comes from a variety of locations is overcome by creating a consolidated plan. First, both your IT and business level leaders must agree to a joint ownership model for collaboration. Next up, you’ll need to select the key integration areas you want to tackle and prioritize them. Then, select a data integration solution that understands what you’re trying to accomplish. Finally, implement an API platform (adding data warehousing when necessary) that doesn’t require your IT team to write new code.
Like so much of our world, integration challenges can be resolved through planning, communication, collaboration and an excellent API platform.