1000 jobs for BigData Analytics posted in 1 week !!

Posted on February 20, 2014 by Subhashini Tripathi | Comments(2)

I teach a BigData Analytics course in Bangalore and I routinely check up for jobs that exist in this domain on Naukri.com. (Naukri.com is the no.1 job site in the country). You must be hearing about BigData , Cloud technologies and Analytics being the ‘hottest’ jobs of the century. Quoting Harvard Business Review ,” Data Scientist: The Sexiest Job of the 21st Century. So who is a Data Scientist ? It’s a high-ranking professional with the training and curiosity to make discoveries in the world of big data. The title has been around for only a few years. (It was coined in 2008 by one of us, D.J. Patil, and Jeff Hammerbacher, then the respective leads of data and analytics efforts at LinkedIn and Facebook.) But thousands of data scientists are already working at both start-ups and well-established companies. Their sudden appearance on the business scene reflects the fact that companies are now wrestling with information that comes in varieties and volumes never encountered before. If your organization stores multiple petabytes of data, if the information most critical to your business resides in forms other than rows and columns of numbers, or if answering your biggest question would involve a

Continue reading…

Facebook Open Sources Presto SQL Query Engine

Posted on November 12, 2013 by Himanshu Sachdeva | Comments(0)

In June 2013 at Analytics @ WebScale conference, Facebook announced Presto which they were using internally to process petabytes of data. It has now been made open-source as per a recent post by Facebook Engineering. So what is Presto? Hive, which was initially developed by Facebook used MapReduce chaining to transform a query into multiple MapReduce Jobs. Presto different as it does not use MapReduce & is 10 times faster that Hive for most queries as per Facebook. Presto allows querying data where it lives, including Hive, HBase, relational databases or even proprietary data stores. You can issue SQL like queries on Presto that include left/right outer join, subqueries or even common aggregate functions. A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization. Facebook uses Presto internally to interactively query over a petabyte of data by about 1000 employees running more than 30,000 queries a day. Currently its also being used by leading internet companies including Airbnb and Dropbox.   You can find more about Presto here : Presto Website Facebook Blog about Presto Gigaom Story

Jobs – Big data and Analytics

Posted on August 20, 2013 by Subhashini Tripathi | Comments(0)

A search on Naukri with key words Hadoop + analytics / Big data + analytics gave me some interesting insights into the type of jobs that are currently floating in the Indian market. My take-away is as follows :- 1. Job Description: – this is very brief . Eg:- Design and Implementation of Big data utilities for Business Analytics Data mining and Explorative analytics for various Industry Benchmark Reports 2. Competency Required :- contains terms of technology understanding , logical thinking and ability to solve unstructured problems 3. Desirable Skill Sets :- Good understanding of Business Metrics and parameters. Ability to apply right statistical tools to business case Technical – Apache Hadoop, Pig, hive, big data Domain Exp MANDATORY 4. Pay – Best in the industry – often 1.5 times that in pure play analytics or Hadoop 5. Experience – 4+ years’ 6. Employer – Top MNC Terms that crop up include Identify pilot opportunities Drive early development Build reference architecture and blue prints Network extensively with the ecosystem, lead and participate in conferences and workshops, and establish thought leadership So what is big data analytics? As a domain, it is very new and encompasses ETL, data visualisation and Statistical analytics on ‘big data’. Big data

Continue reading…