About Author:

Roopa K

Roopa K

Getting Started with Kinesis Firehose

Posted on June 1, 2016 by Roopa K | Comments(1)

       In this fast growing world, humongous amount of data is being produced from all sources in every part of the world. It can be anything like logs from the machines, data produced from the traffic signals, data from the IoT devices, smart devices installed in homes/IT industries and a lot of other sources. After production of this vast amount of data, another problem arises of storing, configuring, managing and streaming of data. How to manage data which occupies storage, utilizes compute power, used for analysis is an important aspect for decision making? AWS has a solution to it. Amazon Kinesis streams is the service that you are looking for to stream the data.        Kinesis Streams will collect data form the source and stream to application for further analysis. The data is replicated across availability zones   for high availability and reliability of data. It can scale based on the incoming data. It can scale from megabytes to terabytes while streaming data. It loads data into stream using HTTPs, Kinesis Producer library, Kinesis Client Library and Kinesis Agent. Basically in Kinesis Streams the data is available up to 24 hours and can also be extended up till 7 days. Kinesis Streams resolved the problem of analysis, compute power and decision making. But, we still have a problem of storing the data. Since Kinesis Streams can only save data up to 24Hrs initially and can be saved till 7 days. What if we need to store the data for long??? What

Continue reading…