Uncovering the Role of Data Scientists in Big Data
Written on
Chapter 1: Introduction to Data Scientists and Big Data
In this rapid course designed for executives and market professionals, we delve into the essentials of Big Data.
Watch the video for a summarized insight before diving into the text.
A notable article in the Harvard Business Review from October 2012 sparked global interest with its title "Data Scientist: The Sexiest Job of the 21st Century." Authored by Thomas H. Davenport from MIT and D. J. Patil, who coined the term "Data Scientist," the piece highlights the emerging importance of this role. Patil later became the first "US Chief Data Scientist" under the Obama administration.
The essence of Big Data lies in extracting valuable insights from vast amounts of data, and such insights require skilled individuals capable of asking the right questions and utilizing Data Analysis effectively. This is where the Data Scientist emerges as a crucial, yet still evolving, profession that educational institutions are striving to cultivate.
In summary, "data-driven companies" leverage data effectively, relying on insights generated by Data Scientists. A Data Scientist's skill set encompasses Programming, Statistics, Mathematics, Machine Learning, Data Wrangling, Visualization, Communication, Software Engineering, and perhaps most importantly, a strong "Intuition" for problem-solving.
Additionally, possessing "domain knowledge," or specific expertise in a certain field, is essential. For instance, a Data Scientist specializing in genetic mapping needs a profound understanding of Biology.
Section 1: The Data Scientist as a Unicorn
Data Scientists are often referred to as "Unicorns," mythical creatures with extraordinary abilities, akin to a horse with a horn that is hard to come by. Similarly, the Data Scientist is a rare find—someone who integrates technology, mathematics, statistics, data science, and business acumen.
Professionals from various backgrounds, such as Physicists, Mathematicians, Economists, and Managers, are increasingly pursuing studies in Data Science to bridge knowledge gaps in Data Analysis and Big Data Technologies, making this one of the most sought-after professions globally. While many may excel in Data Science, they often lack domain knowledge necessary to fully comprehend business challenges.
Over time, educational institutions, including universities and specialized schools, are organizing to provide curriculums aimed at training Data Scientists.
Curiosities
- The difficulty in defining a Data Scientist has led to humorous anecdotes in Silicon Valley.
- One joke defines a Data Scientist as "a Data Analyst who lives in Silicon Valley."
- Another quip states, "A Data Scientist is someone who is a better statistician than a software engineer, and a better software engineer than a statistician."
- For a visual overview of the Data Scientist profile, check out this infographic: http://bit.ly/2F2oDX5
- Data Scientists should possess knowledge in four key areas: 1 — Mathematics and Statistics 2 — Programming and Database Management 3 — Domain Expertise 4 — Communication and Data Visualization
Section 2: Big Data Teams
- Patil famously stated that Big Data is a team sport. To effectively execute Big Data projects, collaboration among Business Analysts, Software Engineers, Programmers, Systems Administrators, Cloud Computing Managers, Product Managers, Designers, Data Visualization Experts, and Software Development Managers is essential. The Data Scientist plays a vital role in this team, tackling real-world problems by generating insights from data.
Often, a Data Scientist takes on multiple roles until a more comprehensive team is established. In some instances, existing IT teams may initially support new projects. In certain scenarios, addressing a particular issue may require hiring specialists in platforms like Google Cloud (BigQuery, Cloud Dataflow, Tensorflow), IBM Analytics, Amazon AWS, or platforms like Cloudera and Databricks with Spark.
Ultimately, building effective teams is crucial for achieving success in Big Data initiatives.
Curiosities
- According to the American job site Glassdoor, the average annual salary for a Data Scientist is $121K, with a range from $87K to $158K. http://bit.ly/2EqjKpq
- In Brazil, Data Scientists typically earn between R$ 6K and R$ 15K, depending on their experience, education, and practical knowledge.
- Companies are establishing new departments (OCDO — Office of the Chief Data Officer) focused on creating innovative solutions and revenue streams from data. CDOs earn competitive salaries in the Big Data realm.
Support the author's work by subscribing to the newsletter. Watch the next lesson video titled "The Main Technologies of Big Data."
About the Author: More information about these articles can be found in the book "Big Data for Executives and Market Professionals — Second Edition." To continue reading, return to the Course Overview and select the link to your next lesson.
This video provides an introduction to Big Data and Data Science, covering essential concepts and applications.
This video discusses the interplay between Data Science and Big Data, highlighting their significance and use cases.