Ever wondered what happens between Map and Reduce?

Shuffle and Sort – The input passed to every reducer is sorted by a key. The process of sorting and transforming the map outputs into reducer outputs is known as Shuffle.

MAP side

The output produced by the mapper is not directly recorded onto the memory. This process involves buffering and processing data further to enhance efficiency. It is often a good idea to compress the map output while writing it onto a disk, as doing so improves performance, saves disk space, and optimizes the volume of data that is being transferred to the reducer. By default the output is not compressed, but it is easy to enable by setting the value of ‘mapred.compress.map.output’ to ‘True’.

Map-reduce-areaReduce side

The map output file resides on the local disk of the task tracker that runs the map task. This requires further processing by the task tracker that is about to run the reduce task for the partition. The reduce task requires the map output for a particular partition from several map tasks across the cluster. The map tasks may complete at different times and the reduce task starts copying their outputs as soon as each map task completes.

Bodhtree, a leader in ‘PACE’ technology IT Services, including Product Engineering, Analytics, Cloud Computing, and Enterprise Services.   Bodhtree empowers innovative businesses strategies through a mission to Educate, Implement, Align, and Secure transformational technology solutions.

Read More

How BI for mobile devices on top of Big Data can transform your employees, customers, and enterprise



BI is going Mobile – in a ‘Big’ way.   Business decisions are not always made in the corner office or cubicle.  They are made on retail floors, in delivery trucks, on an ambulance, or at the laboratory.

Mobile BI delivers intelligence to where ever your team makes the critical decisions.   By delivering the right information to the right people at the right time, mobile BI accelerates response to real-time information and enables a more agile enterprise.

Big data

Big data is exponentially enhancing intelligence quotient of BI by better leveraging all data inside and outside an enterprise; this intelligence can empower business decisions in terms of both timing and reasoning

Combination of Mobile BI and Big Data

Now the combination of Mobile BI and Big Data further reduces the gap between data generation and business decision.

In a recent survey conducted by SAP, 70% of CIOs envisioned killer Big Data apps useful to their enterprises, but interestingly most of them chose not to reveal the idea as it would reduce the competitive advantage.

What types of mobile analytics on Big Data apps could these CIO’s consider so critical to a market advantage?  Possibilities include geospatial intelligence, behavioral intelligence, enhanced customer interaction, etc.

Sample Use case

For U.S. Xpress Inc., a trucking company based in Chattanooga, Tenn., the driver to move to Big Data analytics and real-time BI reporting was a desire to get more out of the large volumes of sensor data being collected from the company’s trucks. U.S. Xpress was looking to use the data to enable its fleet managers to “answer very specific, detailed questions” about trucking operations.

Phani K Reddy is a Big Data Architect with Bodhtree, a leader in Data Analytics, Business Intelligence, and Big Data services. Bodhtree provides Hadoop implementation and maintenance services as an end-to-end service to solve specific business challenges.

Read More

The new era of Cloud Computing

In 1995 “Internet” was the new Buzz word in the business world. Some predicted it would not work, some thought it would; it did and today it is the basis for all kinds of business. So much so that nothing can work with out internet.

With the World Wide Web evolving, web 2.0 and web 3.0 have come into picture. Whats their use What did they get along and what do they have to offer Commom questions indeed. The answer is “Cloud Computing“! Social Media marketing is the new trend companies have been adopting and still are. But during this evolutin, another one creeped up alongside. The companies that use platforms like twitter and facebook for online marketing are now coming up with their own platforms of collaboration. (more…)

Read More