A New Way to Aggregate Data from Silos of Process Information

The ability to extract process knowledge has become easier through the years. Technology has evolved to the point where we can deploy capabilities that connect at multiple levels to extract different types of process insight. In the past, organizations were forced to spend enormous energy extracting data manually from different applications and databases. Then, they would have to use things like spreadsheets to transform the data and convert it into meaningful information. 

These exercises were slow – from start to finish, the journey could take several months just to get to the stage where data was transformed into something that could be analyzed. When dealing with millions of transaction records, this would lead to a high error rate and steps in the process being missed completely resulting in a very incomplete picture of the process environment. By the time that sense could be made from all of the data there was a very good chance that things in the environment had changed so the information gleaned perhaps no longer reflected the current state of things.

The Evolution of Process Mining

Things have now changed considerably. Process information can be gathered from multiple sources. With Process Mining, we now have the ability to automate connecting to databases and application log files. We can then extract transaction information with all of the activity steps and other supporting information. At the desktop level, we can use Task Mining technology to capture step-by-step process information from user activities that do not directly get stored in application databases. 

For example, as part of the process, a user may fill in a spreadsheet and populate it with some important detail which then gets emailed to the HR department for further use. Those activities may be crucial parts of the process that Process Mining cannot pick up on because it’s not stored in a database or other system of record. 

360-Degree Insight Into Your Processes

Task Mining plugs in the gaps from that perspective by grabbing user activities as they switch between different applications and tasks. Test Automation gives you further insight by exposing the process flows that you’re testing as these have been identified as elements of the business that are critical to operations. 

These libraries are often extensive and can provide extensive insight into your processes. Production automation via RPA provides another dimension to process understanding by making information available as to activities within a process that is being automated in production and thus make up a part of a working process today.

The different technologies above provide meaningful process insight in silos. But each of them has weaknesses when approached singularly. While Process Mining can provide a visual representation of your processes that is easy to walk through, what it cannot do is present detailed activities stored outside of a database. 

Task Mining, on the other hand, is great at grabbing activities on a desktop and across applications, but it cannot give you insight into how your processes are performing and show you where the problem areas and bottlenecks are. Test automation only gives you one side of the story, only showing what you are automating in a test and QA environment. RPA is often very focused on automating only singular activities which fit a very specific ruleset, so you often don’t get information on the complete end-to-end process.

Solution: A Multi-Dimensional Approach

The way forward for a more mature approach to process discovery is to have the ability to blend data from all of the silos mentioned above and then present all of the learned data in a holistic 360° view. A centralized view that can present a complete picture by taking multiple sources of truth and then fill the gaps where any might exist, gives rise to true Process Intelligence which is a huge leap forward when compared to siloed information. By adding AI and Machine Learning into the mix, activities between different data sets can be aligned together and a more complete end-to-end picture can be painted. 

Having process knowledge from multiple sources in one repository makes it easier to then start mapping all of the various activities to the organization. What becomes powerful is that you can then get not just a summarized organizational view but also a view per department as well. Department managers then get greater insight into what is directly affecting them and where they have potential areas for improvement within their department. They also get insight into where bottlenecks within the operations of other departments might be affecting them, allowing different roles to collaborate to improve things across the entire organization.