Last week, Lightspeed Venture Partners and the Amazon Web Services team brought together top technologists, startups, students, former Lightspeed Summer Fellows and venture capitalists to discuss how big data trends are impacting companies today. Companies like Boundary, DataStax, Delphix, EdgeSpring, MapR Technologies and Qubole – all working on innovative solutions in big data spanning the underlying infrastructure to analytics – were also in attendance, sharing their insight with others looking to utilize big data to make their solutions possible.
At Lightspeed, we spend a lot of time talking to entrepreneurs about big data and have been investing in companies offering disruptive solutions such as DataStax and MapR, two pioneers building enterprise class platforms for big data workloads leveraging Cassandra and Hadoop. While the big data revolution is still in its early phases of development, we believe that a shift is occurring where there is a significant opportunity for companies to gain a competitive edge by collecting and analyzing full datasets instead of crafting predictive algorithms that rely solely on sampled data. Put another way, the core theme is a simple idea that “data beats math.” We’re starting to see that almost every industry, from IT to healthcare to retail, with companies like Boundary, which uses big data to help companies monitor and improve application performance to deliver optimal performance and uptime.
It was no surprise in speaking with several attendees during last week’s event that the common theme of conversation was taking the progress that has occurred in big data platform technology over the past several years and applying it to solve specific consumer and business challenges – i.e. creating so-called “killer apps.” One such startup we spoke with was Kidaptive, a company that is developing applications for children that are created in conjunction with top university researchers to provide adaptive content that enables learning through games. As children play the games, the technology assesses their performance and provides parents with actionable intelligence and useful tips about how to further support their child’s progress and interests. At Lightspeed, we are bullish on the educational apps space and are investors in Duck Duck Moose, creator of children’s applications for mobile and tablet devices.
However, while there is a lot of excitement around using big data to bring more value to businesses and consumers, Patrick McFadin, a solutions architect from DataStax, points out that there can be a gentle balance involved. When creating these types of next-generation big data solutions, especially when information about specific consumers is being utilized, it will be important to provide them with value without crossing delicate boundaries of privacy and security.
What do you think is the next big thing for big data? How are you innovating to create a big data solution that solves business or consumer challenges? Let us know in the comments, or send me a tweet at @krishparikh.