Happy New Year to you! I spent time over the holidays reflecting on 2018 and planning for 2019. I wanted to share some reflections and some thoughts on what we anticipate in the fast moving world of data lakes for 2019. But before I get into any of that – a big thank you to our customers for having us as their partner in their data lake journeys. Here are some notable things from 2018:
Strong Cloud Data Lake Growth – At a macro level, the growth of data lakes is undeniable. The global market for data lakes is also projected to grow 28% annually from 2017-2023, reaching a total value of approximately $14 billion by 2023. But the world of cloud data lakes is growing much faster and becoming a reality for a lot of enterprises. The key tenets of flexibility and choice are resonating very well. According to AWS CEO Andy Jassy, there are already more than 10,000 data lakes built on top of S3.
Drive to Democratize Data – We continue to see demand from data consumers for greater access to data. Propelling this trend is the need for organizations to empower data consumers with best-of-breed self-service analytics tools to generate more business insights faster. However, this creates a tension between consumption and governance of data. This becomes particularly challenging when an organization uses multiple analytic tools, which has been one of the more common scenarios we’ve encountered in 2018.
Data Usage Regulation Goes Mainstream – Okera exhibited at both Strata NY and AWS re:Invent in 2018, and compliance with legislation like GDPR and CCPA was a hot topic, showing it’s clearly on the radar of data professionals. In our conversations with trade show attendees, we’re seeing companies fall into one of two camps: those that have been proactive implementing data governance, and others who are holding off on investing more in governance until there’s clarity on the requirements. Regulations are evolving and more are coming in. This ambiguity and need for flexible and scalable governance is hindering implementations, creating a risk for organizations that can impact shareholder value.
What’s Ahead in 2019
- Maturation of Cloud Data Lakes. Architectures and best practices for cloud data lakes aren’t well established just yet. The notion of a data lake being a capability for the enterprise will become more mainstream and architectures and best practices will emerge to enable data lakes. An important aspect to enable the overall capability is the need for data fabric technologies that enable the various technology components to work together. For example, a federated data lake in AWS, comprising of several accounts and S3 buckets needs a data fabric that makes them all work together in a secure fashion across multiple analytic tools.
- Containers and deployment management software like Kubernetes have made service oriented architecture and microservices approachable for development teams. Using an agnostic abstraction layer like Kubernetes also makes portability easy, thereby enabling multi- and hybrid-cloud environments real. In 2018, we saw some active development in this space and it intersecting with data architectures. I believe we’re going to see a lot more movement in 2019 and more and more enterprises are going to want to drive towards a service oriented and portable data lake architecture.
- Machine Learning engines are becoming feasible for the mainstream enterprise. We’re going to see creation and commercialization of more engines that are now possible to run because of the scale with which you can store and compute data. The creation of more engines exacerbates the problem of lack of standardization of model management, deployment and monitoring. We’ll likely see some work on this front this coming year.
- Data Usage Regulations – 2018 saw a lot of activity and noise around GDPR leading into the May 25th deadline. The real enforcement of the regulation is ongoing and May 25th was just the day from which people had to start complying. Similarly, there are other regulations that are going to matter very soon, such as CCPA. Not only will the clock start ticking for CCPA at the start of the new year, but there’s a real possibility new PII legislation may emerge at the Federal level in the U.S. which could supersede CCPA. For organizations, this means making sure data is used in the right manner and there is visibility into the usage. I believe we are headed into an era of more regulations around data usage in different contexts. 2019 will see some development on this front.
Okera 2018 Milestones
At Okera, we’ve had quite a year. We’ve more than doubled in team size, brought our first product to market, have successful customer deployments at large scale and have a lot of encouraging traction with new prospects. Some of the key highlights are:
- Okera emerged from stealth mode in 2018, announcing the general availability of our first product, the Okera Active Data Access Platform.
- Since our 2018 launch, we successfully deployed at multiple FORTUNE 500 companies. We actively collaborated with early customers to build the right product – one that’s fast, that scales and works with ease. We’ve come a long way in this year and still have a lot of ground to cover. We also hosted our first customer advisory board where we had majority of our customers as well as some prospects attend and give us feedback on what they care about.
- Product Innovation – Following GA release, our engineering has kept an aggressive pace of product development, introducing new capabilities including: support for Snowflake, Redshift, and relational databases, interactive reporting, advanced access control, and intelligent schema management. Expect to see even more innovation from Okera in 2019.
Looking for a New Opportunity in 2019?
- Okera is Hiring – Speaking of innovation…if you’re looking to join a fast growing team that thrives on it – we’d like to hear from you! Check out Okera’s career page to see if one of our engineering, sales, or administrative opportunities might be your next big opportunity.