Is being a data scientist stressful? Listen to them directly

Data scientists have a time when their work is one of the most sought after and highest paid in any industry. And the demand for positions that require data science and analytical skills continues to soar. According to an AIM report, there were nearly a lakh of data science-related positions available in India in 2019 alone.

This demand is the result of the proliferation of data and the potential it holds in the business world. This created a pool of professionals eager to acquire data science skills for a potentially lucrative career. But there is a high cost: high stress.

Whether you’re drawn to the prospect of financial rewards or looking to solve real-world problems with data, it’s important to prepare for a future that can be fraught with stress. Six data scientists spoke:

Problem solving and troubleshooting

By having a wide range of roles that involve working on large data sets, you prepare for periodic disruptions that occur in a consistent fashion. And often it takes a long time to find or fix a discernible source of the problem and it can happen in a continuous loop.

Similar problems may be common in the banking industry, according to data scientist Usha Rengaraju. “Models that work great locally may not work the same during piloting or while scaling. Plus, in such situations there is always a lot of pressure to find fixes quickly, ”she says.

Likewise, you might have an algorithm that doesn’t quite work as you see it, or you might be sitting with a conclusion that isn’t consistent across the board as it should be.

Data gathering

Before you can dive into the more interesting aspects of the job, you need to collect data, and this stage can be particularly stressful. “To access the data, I have to coordinate with multiple product teams and get the relevant approvals,” says Surya Prakash Manpur, who works as a Data Scientist at RealPage.

In Rengaraju’s opinion: “Working with large companies to collect data from multiple departments of the client company can be quite stressful. “

However, care should be taken at this point, as your analysis will only be as accurate as the quality of the data you collect. If you devote yourself to perfecting a report whose data itself is unreliable, your report will not be effective.

It could also get a bit complicated since you have very little control over the raw data of the company and thus testing its veracity becomes a problem.

“The next step is to work with the developers to extract the quality dataset and understand the business implications of the dataset, which can also be stressful processes,” adds Manpur.

System-wide collaboration

There was unanimous agreement from most of the data scientists who participated in this story regarding the stress that collaborating with stakeholders can create.

“Data scientists need to understand many diverse perspectives from many stakeholders and communicating the impact of analytics to them can be difficult,” says Navin Manaswi, Data Science Specialist at WoWExp. “Balancing this act could be a stressful business, especially for people who are more introverted by nature,” he adds.

According to Rengaraju, negotiation, communication and time management are as important as programming and analytical skills for a data scientist. In addition, the high expectations that customers place on them can create a lot of stress for them.

“Understanding the needs of the business and communicating a metric or KPI to them can be quite stressful,” says Manpur. “The ability to say no is another issue. There may be situations where the expected history of the trading partner conflicts with the information extracted from the analysis performed on the dataset, ”he adds.

Paradoxically, despite the high level of collaboration involved in the work of a data scientist, the truth is that you will be spending a lot of time working alone, especially in large organizations.

“To compensate for this, our sales and scientific teams are working in tandem in an agile manner,” says Vidhya Veeraraghavan, chief analyst at Standard Chartered Global Business Services. “It also helps the data science team collaborate and help each other deal with the stresses of the day,” she adds.

High expectations of data scientists

High performance expectations due to the deep nature of some of the issues you solve make this role difficult and stressful. Data scientists typically work on an entire company’s data, which means they’re going through thousands of transactions at a time.

Your analysis has the potential to influence key decisions made by a business, and that in itself can be a lot to deal with. But Sray Agarwal, AI and ML specialist at Publicis Sapient, has learned to accept the nature of work.

“Data science is more exciting and adventurous than stressful,” he says. “It’s only stressful when you’re working to pay bills, not solving real-world problems,” he adds.

Increased competition among aspiring data scientists

Data scientists often have to put in long hours, especially when working to solve a big problem. But the field has become very competitive over the past few years, and the very level of competition can be stressful.

This requires staying on top of the competition and learning more.

“I find the acquisition of knowledge in a field very stressful, especially in niche fields like genomics, geology and semiconductors, etc. Says Rengaraju. Tribhuvan Tewari, Data Scientist at Stratbeans, adds Tribhuvan Tewari: “Keeping up to date with new technologies and the corresponding market requirements is also a constant stress.

Sean N. Ayres