_edited_edited.png)
BIG Data
.. is a combination of structured, semi-structured and unstructured data collected by organizations that can be mined for information and used in machine learning projects, predictive modeling and other advanced analytics applications
Big data can be from various sources like transaction processing systems, customer databases, documents, emails, medical records, internet clickstream logs, mobile apps and social networks.
Big data is often characterized by the three V's:
· the large volume of data in many environments.
· the wide variety of data types frequently stored in big data systems; and
· the velocity at which much of the data is generated, collected and processed
Some people ascribe even more V's to big data like Veracity (Credibility or trustworthiness), Value (business value) and Variability (it’s format and how to use).
Companies use big data in their systems to improve operations, provide better customer service, create personalized marketing campaigns and take other actions that, ultimately, can increase revenue and profits. Businesses that use it effectively hold a potential competitive advantage over those that don't because they're able to make faster and more informed business decisions.
For example, big data provides valuable insights into customers that companies can use to refine their marketing, advertising and promotions in order to increase customer engagement and conversion rates. Both historical and real-time data can be analyzed to assess the evolving preferences of consumers or corporate buyers, enabling businesses to become more responsive to customer wants and needs.
Big data is also used by medical researchers to identify disease signs and risk factors and by doctors to help diagnose illnesses and medical conditions in patients. In addition, a combination of data from electronic health records, social media sites, the web and other sources gives healthcare organizations and government agencies up-to-date information on infectious disease threats or outbreaks.
Here are some more examples of how big data is used by organizations:
In the energy industry, big data helps oil and gas companies identify potential drilling locations and monitor pipeline operations;
Financial services firms use big data systems for risk management and real-time analysis of market data.
Manufacturers and transportation companies rely on big data to manage their supply chains and optimize delivery routes.
Other government uses include emergency response, crime prevention and smart city initiatives.
Big data and Data Analytics intersection
The main aim of big data is to convert the raw data into meaningful data sets that can then be used for drawing meaningful insights or solving complex business problems.
Meanwhile, data analytics deals is mostly with structured data. It analyses the structured data to answer complex business queries, find solutions to business challenges, etc.
Since big data is a more comprehensive and extensive process, its tools are complex and sophisticated. Tools such as automation and parallel computing tools are used to convert unstructured data into meaningful data sets. Meanwhile, simple tools like statistical modelling and predictive modelling are used in data analytics. Moreover, many statistical and mathematical formulas are used while analyzing and interpreting data.
Most tools designed for data mining or statistical analysis tend to be optimized for large datasets. In fact, the general rule is that the larger the data sample, the more accurate are the statistics and other products of the analysis.
Recent generations of vendor tools and platforms have raised the performance applications involving big data. Most modern tools and techniques for advanced analytics and big data are also very tolerant of raw source data (non-standard and poor-quality data). Putting Big Data and Advance Analytics together provide new insights to improve business outcomes.
Data Analytics
… is process of converting raw data into actionable insights, by examining data sets to find trends and draw conclusions about the information they contain. It involves using range of tools, technologies, and processes used to find trends and solve problems by using data. Data analytics can shape business processes, improve decision-making, and foster business growth.
There are four key types of Data Analytics · 1. Descriptive Analytics · 2. Diagnostic Analytics · 3. Predictive Analytics · 4. Prescriptive Analytics. While the first two focus on providing insights on past happenings, the later two types focus on providing guidance on future possibilities
Let us take a look into the differences amongst these 4 types and the relevance of their application.
Descriptive Analytics looks at past performance and understands the performance by mining historical data to identify the cause of success or failure in the past.Almost all management reporting such as sales, marketing, operations, and finance uses this type of analysis.
Diagnostic Analytics examines data or content to answer the question, “Why did it happen?” It is characterized by techniques such as drill-down, data discovery, data mining and correlations.
Predictive analytics is the process of using data to forecast future outcomes. The
process uses data analysis, machine learning, artificial intelligence, and statistical models to find patterns that might predict future behavior
Prescriptive analytics, meanwhile, takes this one step further by identifying one or more actions that an individual or organization can take in response to a given forecast. Prescriptive analytics also attempts to determine what outcomes these actions may lead to in their turn.
Application of Data Analytics
Data analytics is instrumental in delivering improved business outcomes in various areas of applications. Numerous packets of data are circulating all around the world due to increasing access to the internet. Businesses are aware that this data translates to information which they can use to improve their customer service, understand trends, or even find market loopholes. Not only businesses but even civic bodies are using data analysis for several reasons. Different areas where data analytics is currently applied are across the globe are
1. Security - to drop crime rates in big ciites
2. Transportation - for last number of people in events like Olympics
3. Risk detection – by banks for understanding probability of customer defaulting
4. Risk Management – by insurance companies to analyse claims data and plan operations
5. Delivery - by companies like DHL and FedEx to provide cost effective delivery
6. Fast internet allocation – for smart allocation of bandwidth with understanding on how bandwidth is being used in specific areas and for the right cause
7. Reasonable Expenditure - by govt bodies in directing the tax money in a cost-efficient way to build the right infrastructure and reduce expenditure
8. Interaction with customers – by insurance companies to establish healthy relationship between the claims handlers and customers by understanding demographic preference of channels
9. Planning of cities - data analytics would help in bettering accessibility and minimizing overloading in the city
10. Healthcare - to optimize the cost of health care services with improved efficiency of using machinery, medicines etc.
11. Travelling - identify desires and preferences of different customers to help optimizing the buying experience of travelers, by allowing to customize their own packages
12. Managing Energy - like smart-grid management, optimization of energy, energy distribution, and building automations
13. Internet searching - Most search engines like Google, Bing, Yahoo, AOL, Duckduckgo, etc. use data analytics with different algorithms to deliver the best result for a search query, and they do so within a few milliseconds
Due to the sheer size and variability of big data, data analytics is done with the aid of specialized systems and software. There are multiple tools offering different advantages for data scraping to analysis, Statistical analysis and data mining, Sharing code, creating tutorials, presenting work, Big data processing, machine learning, business intelligence, multivariate and predictive analysis, data visualization, like creating data dashboards and reporting.
In conclusion, business intelligence has come a long way from the traditional ways of working. Traditional ways/tools are constrained by IT in managing, adding, and changing data sources time consuming. Big data combined with modern IT, and analytical technics and tools empowers organization to leverage 100% of its available data across 100% of the business taking the organizational performance outcomes to next level.