Turning billions of data points into proven business opportunities
Our client’s backstory
Our client is a market leader in sleep innovation and bed manufacturing, powering the sleeping experience with smart bed technology to enhance users' sleep and mental well-being.
Besides serving 1+ million satisfied customers, the company counts thousands of employees who track all the different processes, from sales to delivery. All operations are run on several large enterprise systems that track inventory, transportation, consumer insights, and more. Each of these systems, however, solves a narrow domain of problems and functions as an isolated silo of data.
To break down the silos between different departments and enable a complete overview of user/sleeper activity all in one place, the client decided to build a new platform where all user and company activities would be available in one place.
Since Klika has been working with the client for years on almost all parts of their systems and applications, we were entrusted with building the enterprise data platform to unify all operations, activities, and device information under one platform.
Challenges
We first had to identify sources and types of data from the immense load of data generated from IoT devices, millions of users, and company departments. The next step was to define the parameters that would separate irrelevant from relevant data, which is a challenge when dealing with large datasets.
Data ingestion and storage, most conveniently and cost-effectively, had to be considered alongside a smooth architecture up to the highest security standards and easy to navigate, given that it was meant for both IT and non-IT departments.
Klika solution
The central part of the EDP was built around data ingestion coming from different internal and external platforms and applications. We ensured that all data from customer orders, user profiles, and net promoter scores were reported directly on the platform, along with third-party gadget data from devices such as Fitbit, Nest, and Armour.
Most of the incoming data was used for analytics and reporting to various departments: the Product Team, Marketing Team, Customer Insights, and Sales Team. Dozens of automated reports for each team were implemented, ranging from simple Excel reports to Power BI real-time dashboards.
We relied on data factories for ingestion purposes. Some were batch jobs triggered per time interval, and others were streamed through event hubs and streams for near real-time ingestion.
We used HDInsights, Apache Spark, Apache Hive, Apache Kafka, and HBase regarding technology. Apache Hive was our primary tool for querying vast amounts of data and building reports, but other tools and storages were also used (SQL database and ComsosDB).
The largest portion of data comes from users; sleep sessions are measured around the clock. The system stored 1.2 billion sleep sessions from around 1 million beds in use.
Our QAs ensured end-to-end testing was carried out throughout the project at all development stages.
We successfully made the enterprise platform a centralized access point for all internal and external data sources. Connected to a large number of IoT devices, it provides a vast amount of data that is used not just as a data source for different sales and marketing reports but also as a foundation for building other user features, like integration with activity trackers (Fitbit, Under Armour, Nest).
Results
The enterprise platform created endless possibilities for data management and insight – breaking down organizational silos and allowing for informed decision-making across all company teams.
It increased efficiency across teams who are now faster to respond to client inquiries, anomalies, or risks thanks to real-time data insights at any moment. The company started using the platform as a source of truth to develop business strategies and identify emerging and future needs. The Klika team is still part of the EDP project, taking care of updates and maintenance.
Technology stack
SQL, ComsosDB, HDInsights, Apache Spark, Apache Hive, Apache Kafka, HBase
