Decluttering R

DSC_0937
Importance of decluttering the R environment

R is a versatile and powerful programming language that enables the user to perform various types of statistical and data analyses. Like with any other tool, R’s potential largely lies in the user’s knowledge of the extent of its capability. Having used R extensively over a period of time, we have some useful tips we think will benefit the beginner and the seasoned R user alike. Because R is open source, its adaptation has increased exponentially. Several users without any programming or computer science background have been able to benefit from it. Being a newcomer to programming and scripting languages myself, I have fallen prey to several programming and scripting fallacies. Over the course of time, thanks to a multitude of help from experienced colleagues, and to the sea of information readily available on the internet, I have been able to learn several programming etiquettes which I wish I knew sooner. Continue reading Decluttering R

Dispelling illusions using Visualizations

Visualizations are a great data exploration technique. Our human minds are better able to understand and retain visuals than scripts or text. Visualizations, apart from giving us a good general overview of the data, entail us with an intuitive understanding of the distribution of the dataset and its trends.
Continue reading Dispelling illusions using Visualizations

The Importance of ‘1’: A Different Perspective on Data Science

As data scientists, we are always looking for data, more data, different tools, or new techniques. We develop models enabling us to find higher areas of crime, make our society safer, or find ways to assist companies increase their profits or find efficiencies. Data scientists can help us identify patterns to determine what customers will buy, when they will buy it and where it will be bought. It can even assist the customer in making suggestions for cross-selling and up-selling opportunities and determining what customers will buy before they even buy it. The capabilities and opportunities of data science are endless and its uses are boundless.

Data scientists can easily forget the true nature of the data, since the massive amount of data available and the complexity of the techniques clouds each observation. Depending on the dataset, every single observation represents a human being, or a living being. Statisticians and data scientists have always referred to the the size of the sample as ‘n’, for example n=100, meaning 100 observations. However, when looking at large amounts of data, it obscures the most important ‘n’, n=1. ‘N’ equals one (N=1) could be you, your spouse, your friend, a sister or brother, a child or parent. It can be someone you know, or a friend of a friend. It is not uncommon for many data scientists to be working with a dataset and realize, that one of the observations refers to themselves.

When we analyze data, of course we analyze the numbers as they are, but we should inspect and respect the data, not as numbers but as human beings, as members of our community, or as a precious life. Of course we can de-identify the data as a means of protecting privacy, the fact that they represent a fellow human or even another life, such as a dog, cat, or other animal, cannot be ignored and should not be considered contrary to our mission as data scientists.

Data scientists must strive to conduct their analysis under a strong ethical code

When we apply this consideration to data science, I believe we are embarking on a new, moral, ethical branch of data science, which can be called Neohumanist data science. As data scientists, we are given an awesome responsibility to see the environment from a different lens. We are entrusted with the knowledge of how to find the proverbial needle in a haystack, and seek truth in the cloud of information. The decisions made from data science impact society as a whole and can greatly help our community, our country or our planet. Understanding the importance of the findings uncovered and its’ impact on the lives of others, therefore, becomes an entrusted gift, when we work with an unbiased perspective and a goal of finding the truth, wherever it may lead.

Data scientists, statisticians and business analysts should always strive to learn new techniques and perform the analyses requested. However, they should always maintain a moral compass that grounds them with a perspective of their responsibility to N=1. They must strive to conduct their analysis under an ethical code that prevents them from deliberately avoiding finding a preconceived truth to further a cause, regardless of the cause. They must never allow themselves to fall victim to Mark Twain’s statement that there are “Lies, damn lies, and statistics”. Becoming a neo humanist data scientist means they will always try and hold themselves to a standard unparalleled in our society. The knowledge, the data, and the tools provided are a gift, of sorts, and they are entrusted to data scientists to make sure that their work will cause no harm to any person, or living thing.

Latest News, Insights and Trends in Analytics and Data Science