About Author:



CloudThat is a leading AWS, Cloud and Big Data training and consulting company. We have trainer over 1500 people in these technologies. Our consulting clients are some of the biggest companies in the world. To learn more visit www.cloudthat.in

Getting Started with Kinesis Firehose

Posted on June 1, 2016 by CloudThat | Comments(1)

       In this fast growing world, humongous amount of data is being produced from all sources in every part of the world. It can be anything like logs from the machines, data produced from the traffic signals, data from the IoT devices, smart devices installed in homes/IT industries and a lot of other sources. After production of this vast amount of data, another problem arises of storing, configuring, managing and streaming of data. How to manage data which occupies storage, utilizes compute power, used for analysis is an important aspect for decision making? AWS has a solution to it. Amazon Kinesis streams is the service that you are looking for to stream the data.        Kinesis Streams will collect data form the source and stream to application for further analysis. The data is replicated across availability zones   for high availability and reliability of data. It can scale based on the incoming data. It can scale from megabytes to terabytes while streaming data. It loads data into stream using HTTPs, Kinesis Producer library, Kinesis Client Library and Kinesis Agent. Basically in Kinesis Streams the data is available up to 24 hours and can also be extended up till 7 days. Kinesis Streams resolved the problem of analysis, compute power and decision making. But, we still have a problem of storing the data. Since Kinesis Streams can only save data up to 24Hrs initially and can be saved till 7 days. What if we need to store the data for long??? What

Continue reading…

Configuring VPN between the VPCs across regions/accounts

Posted on April 1, 2016 by CloudThat | Comments(1)

As AWS is the top player in cloud market, we see most of the organizations migrating their architectures to AWS. If you are familiar with AWS, you must be knowing that most of the services are region level like EC2, VPC, etc. Let’s say you have a complex architecture with instances running in different regions. In case you want your infrastructure spread across multiple regions, securely communicate with each other through an encrypted channel, a secure VPN tunnel would be the need of the hour. How would you plan and create a secure tunnel to connect multiple VPCs in different regions/account, which allows instances in different VPCs to seamlessly communicate with each other using private IP addresses? One of the best solutions for the above scenario is Openswan. Openswan is an opensource IPsec VPN implementation on Linux. For those who are new to networking, VPN stands for Virtual Private Network, used for connecting one or more networks. All data transferred through VPN is encrypted using different kind of encryption algorithms. Openswan uses IKE protocol for negotiation and authentication, packet encryption and decryption is performed by Linux kernel. In order to create a VPN tunnel to connect multiple VPCs, we use

Continue reading…

Track your resource configuration changes with AWS Config

Posted on March 23, 2016 by CloudThat | Comments(0)

AWS Config provides the complete visibility over the deployment and tracking of resources. It checks the inventory changes and identifies the deleted resources. It analyses the compliance of the desired rule against the deployed configuration and respond to security incidents without distorting end user. It helps in troubleshooting the misconfigured resources. AWS Config enables users to get a complete view of the configuration of AWS resources in associated with your AWS account. This enables to understand the relationship among all the resources and their configurations. AWS Config uses AWS CloudTrail to record configuration changes and interdependencies of resources. Why AWS Config ? In AWS Config rules, IT admin provides desired configuration settings for all your resources. These rules are compared with the current deployment to provide an analysis. This result shows how your current deployment is configured and how it should be configured. Sometimes, conflicts occur between the configurations showing the noncompliance in the configuration. This helps the administrators to figure out the misconfigured resources and fix it. What is AWS Config Rules? An AWS Config rule can be explained as the desired configuration setting for specific AWS resources or for an entire AWS account. AWS Config captures configuration changes

Continue reading…

Understanding Amazon Cognito Authentication

Posted on January 8, 2016 by CloudThat | Comments(0)

What is Amazon Cognito? Amazon Cognito is a service which provides user-data synchronization and unique identifiers for your end users that helps you securely manage and synchronize app data for users across their mobile devices. You can create unique identifiers for users through a number of public login providers (Amazon, Facebook, Twitter, Digits, Google or any OpenID Connect are compatible provider) or using your own user identity system. It also support unauthenticated guests, as they might do when they first try an app. Amazon Cognito allows you to securely store and sync data to cloud for these users even though they have not logged in. Amazon Cognito Authentication In order to access AWS resources, a valid AWS credentials (Access Key and Secret Key) need to be passed for authentication. While passing an AWS credentials, security is always a concern. In order to remove the problem of security of credentials, Amazon Cognito creates a unique identifiers for end users that are kept consistent across devices and platforms. It also provides temporary, limited-privilege credentials to access AWS resources. Basically there are three flows of authentication: Basic (Classic) Flow There are three steps to get credentials using cognito: GetId: It is the first call necessary

Continue reading…

Recover EC2 Windows Password

Posted on November 5, 2015 by CloudThat | Comments(1)

There are scenarios where we lose/forget our windows password and then trying out a way to login to the windows server. This turns out to be critical if it’s an admin credential that is lost. But still there’s a way to reset the password. Here’s how to do this: Before going through the steps, we assume that you have a running AWS EC2 Windows instance. For resetting the password we require a Linux machine, which we prefer is an Ubuntu Server in the same AZ as your windows instance. This method requires your EC2 windows instance to be taken offline for some time.

Integration of Azure Audit Logs with Power BI

Posted on October 20, 2015 by CloudThat | Comments(0)

Azure audit logs allows you to view operational logs such as actions performed via Azure Resource Manager, actions that are related to managing resources in your Azure subscription. In order to visualize and analyse these logs in much better way and also to share these information among co-workers Power BI is used.  Azure Audit Log Service is supported only in Azure Preview Portal. The Azure Audit Logs contains the following information for each operation, The name of the operation Level [Critical, Error, Warning] Status [Succeeded, Failed] Resource Time These information can be visualized in Power BI. It is an online Service offered by Microsoft, which is used to collect and visualize data from different sources. This online service can be accessed with your Microsoft work or school account. In Power BI, the data can be either directly imported or through Content Pack Library. You can even create your own content pack for your organization i.e. you can create the report in Power BI and distribute among your co-workers as a content pack. Some of the Power BI services are Azure Audit logs, Azure Mobile Management, Google Analytics, GitHub, etc. The Power BI Azure Audit Logs Content pack is used to

Continue reading…

AWS IoT: A Service for Internet of Things

Posted on October 12, 2015 by CloudThat | Comments(0)

The AWS re:Invent keynote session by Dr. Werner Vogels (CTO, Amazon.com) on day 2 unleashed a new set of possibilities to the world of Information Technology. Imagine Virtualization clubbed with Internet of Things (IoT) and then brought in front of you as a service. And this is now a reality since AWS has launched a new service called AWS IoT, which is currently available in beta version. When it comes to possibilities, AWS customers like BMW and John Deere showed how this service is being used by them to derive meaningful insights from their data in real time. In this blog, I’ll be describing AWS IoT in detail. What is AWS IoT? AWS IoT (Internet of Things) is a service by Amazon Web Services, which helps various devices to connect to AWS and upload their real-time data to AWS Storage, Analytics, Database Services like S3, Kinesis, DynamoDB etc. The data is then processed and mapped to a running application. This feature is also available even when the device is offline. AWS IoT provides a secure way to connect and manage the devices at any scale. It can be leveraged through: Devices: AWS provides SDKs for devices which can be used

Continue reading…

Amazon Inspector- Application Security Service in AWS Cloud

Posted on October 8, 2015 by CloudThat | Comments(0)

The AWS re:Invent keynote session by Andy Jassy (Senior VP, AWS) on Day 1 comprised of announcements about the new services and enhancements in some of the existing ones. In this blog, I am going to talk about one of the new services- Amazon Inspector.   What is Amazon Inspector? Amazon Inspector is an automated security assessment service which identifies the potential security and compliance vulnerabilities in the applications running on AWS. This may be beneficial as a lot of organizations may not have dedicated personnel or staff to identify security vulnerabilities or they might have been overlooked some while testing.   Why is it required? Security is one of the main concerns for organizations whether they are moving to cloud or are already using it. Inspector helps to improve the overall application security by examining it when an application is in production or is being developed. It also helps in adhering to an organization’s security standards and uses AWS’s Security Expertise where the security best practices and rules are constantly being updated by AWS, so one gets the best of both the worlds.   How it works? Amazon Inspector performs an assessment and generates a report containing steps for

Continue reading…

AWS Identity & Access Management – Best Practices

Posted on September 9, 2015 by CloudThat | Comments(1)

  Security is a critical aspect for any organization. This blog focuses on the account security measure provided by AWS – IAM. IAM stands for Identity and Access Management and is used for controlling access to AWS services and resources. There are no additional charges for using IAM. For people new to IAM, the basic concepts are: User: A user is similar to a login user in various operating systems like Microsoft Windows. A user can log in to the AWS console using their username and password. In AWS world, this user can be an individual, system or an application requiring access to AWS resources and services. Groups: A group is a collection of users. Instead of assigning similar permissions to multiple users individually, a group can be created with a set of permissions and users can be added to it. The benefit of creating groups is that it simplifies the tasks of managing a large number of users and their permissions. Role: A role is a set of permissions required to make AWS service requests. But this role cannot be directly assigned to a user or group, instead roles can be assumed by a user, an application or an AWS service like EC2 to make service

Continue reading…

Migration of infrastructure from EC2 Classic to EC2 VPC

Posted on September 7, 2015 by CloudThat | Comments(0)

EC2-Classic to EC2-VPC Migration Your AWS account might support both EC2-Classic and EC2-VPC, depending on when the AWS account was created and regions used. AWS accounts created after 2009 do not support EC2-Classic platform environment and have the EC2-VPC environment. EC2-VPC environment has additional advantages over EC2-Classic environment. In terms of security, VPC has Network ACL which can allow or deny access to a particular IP. Also, we can setup openVPN and customer gateway between VPC and on-premises. This blog will tell you how to migrate instances (both EC2 and RDS) from EC2-classic environment to EC2-VPC environment with zero downtime. Let’s assume that I have my application server running in the cloud infrastructure. The following   architecture diagram can represent the infrastructure running in EC2-Classic environment.   As you can see in the diagram, there is a Route53 Entry for www.mysite.com with ‘A’ name record.  There are two app servers running which are under a load balancer which are pointing to the MySQL RDS instance. In order to migrate the above EC2-Classic environment to the EC2-VPC environment without downtime following steps can be used- Creating a Load Balancer inside the VPC. Creating AMI of app server Launching application server into public

Continue reading…