Learning What Really Matters – Math and Stats
Data analytics is full of tools, and they even change all the time – but rock-solid basic knowledge is evergreen
When checking the requirements of data analytics jobs, some tools are mentioned all the time. They vary a bit, depending on geography, and such as Qlik Sense is very popular in Sweden, where it was originally invented, while Power BI, like everything else from Microsoft, is dominating Denmark.
Databases, tools for transforming data, and programming tools such as Python and R are also asked for, but it varies what exactly is considered good to know. For instance, while R is traditionally considered a very strong tool for statisticians/data scientists, I have seen it mentioned as “old-fashioned”, where they wanted candidates for the job to know the more “modern” Python instead.
Of course, much of this is based on prejudices and lack of knowledge, but it it also indicative for the trends that constantly change in the IT sphere – and data analytics is part of the IT sphere, even if some would like to think of it as something separate – trends, which lead to changes in which an ever-changing stack of tools are utilized.
I have previously mentioned how some BI tools became completely dominant early in the BI era (which was from around the year 2000 and continuing perhaps 10-15 years in most places) – Cognos, BusinessObjects, and Hyperion were the biggest for a while, with a few others trying to catch up.
Looking at Gartner “maps of quadrants” from the 2000s, which were very popular back then, telling which products of different kinds were the most popular, which were runner-ups, etc., the charts of BI products were filled with products that are hardly known to anyone anymore.
So, when Qlik still survives in Sweden, and perhaps in some other places too, it is actually to be seen as an exception. Most other products from then are gone – typically because they were bought by bigger companies and then closed or integrated into their own products.
Power BI has seen a rise, much in the same way as most other products from Microsoft, as a result of “we have it already” thinking in the companies. Whatever you buy from Microsoft, you also get various other products along with it, or, if your are a company, you pay by some kind of points that you have bought, and those points then also give you the right to buy something more at a low price or, at least, without having to ask internally for a separate budget for the new product.
So it’s easy to buy, simply.
Whatever might be the reasons for all the other products of today to have become popular, is sometimes difficult to uncover. There is an open source trend, that’s for sure, and there is also a drive toward using something else than what was used by the previous generation – just because. There’s not always any particular reason for it, other than that the new products are new.
Whatever is the case, they change all the time. And there are many of them.
As a data analyst or anyone else working in this sphere, no matter your exact job title, it can be tempting to try to catch up with all of these tools, in order to be as employable as possible. That, on the other hand, will not work well for most of us. Even if we do manage to get all around and learn how to use every thinkable tool, it will be shot down by the employers when they ask for x years of experience with the tools of their choice. You cannot have several years of experience with everything, so all the stuff you’ve learned will, mostly, represent wasted efforts, as it will not bring you a job.
However, there are some topics that are changing less fast, and where it makes sense to do an effort to become good, because it will be useful no matter which of the tools you end up working with.
These are, among others, mathematics and statistics. Especially the latter, but some mathematics knowledge is needed for understanding the statistics, so they go together.
If you do not have a degree in either, you will probably be seen as inferior to those who have, no matter how good you get. There is such a level of non-flexible thinking present with most employers. I guess it is easier for them, internally in the company, to justify hiring one with a diploma in something than one who is simply good at the same stuff but lacks the diploma.
But even if you’ll be second rank, it is still better to know about these things than not. And if you indeed have a degree, you still might benefit from refreshing your memory about integrating and Gaussian distribution, and all the other goodies, because, as Mike X Cohen, PhD points out in his book “Calculus Unraveled”:
“I learned, forgot, and re-learned calculus several times in my life”
It is like this for all of us – some topics are difficult to remember in detail, especially if you do not use them at all, or not in the same way as you originally learned them.
Mike is very good at everything mathematics and statistics, so if he can say something like that, I guess it really indicates that it is a general phenomenon.
We can all benefit from running through the stuff once in a while, to re-learn or, at times, learn for the first time what we didn’t understand the last time we tried, or just didn’t pay enough attention to, to learn it properly.
For this reason, I have tried, on my new website for data analytics, Inidox Data, to list some links to useful books and courses, and not only for the hardcore data analytics topics and tools, but also for exactly mathematics and statistics. Because, we can all need to look at it again from time to time, and then it’s nice to have such a list at hand, so we can start our search with something that has been looked at already, by someone in the same situation.
There are several lists of different things, and they will be built out and adjusted regularly. They will, however, stay at a limited size, which is simply because I want them to be a useful place to start your search, not to show everything that exists. In fact, there are so many books in the world about these topics, that I would never get to the end of listing them, if I would try.
Some have been left out, even if I have them on my bookshelves and like them. For instance, the book “Stats: Data and Models” by Robert D. De Veaux, Paul F. Velleman and David E. Bock – it is an excellent book, used by many university professors for teaching statistics, and it is readable, complete, full-color, and with many interesting exercises. But it is also very similar to the book that I did include, so I found that there was no point in having both of them there: “Statistical Techniques in Business & Economics” by Douglas A. Lind, William G. Marchal and Samuel A. Wathen.
So, why pick exactly the one I did? Because the first one is from Pearson, who tends to integrate books with the Internet, and permit access to the Internet material only for registered students at university courses, and only for the duration of the course. So, if you buy that book for self-study, you’ll have to use it without the parts and tools and exercises that exist on the Internet and are unavailable to you. They would in any case cost extra money, making the book with tools extremely expensive, all-in-all, so also for that reason, the other book, from McGraw Hill, is a better option. It can be used on its own, but will benefit from having tools like Excel, Minitab, etc. at hand. Some exercises are directly made for one of these tools, and if you don’t have it, you’ll need to improvise or perhaps just skip that exercise.
In general, books meant for university use are often problematic in this way. They require access to some of the information that is typically not available to people outside of the university world: academic papers, etc. (perhaps not as relevant for exactly math and stats, as compared to such as history, but still), and software that is made available to students for a low price, or completely for free, but costs a lot for the rest of us – such as Matlab or Minitab.
For that reason, many of the books I put there as suggested starting points, are open source or free books of some kind, which makes it much easier to get started with them.
There is a third category of books, though, which are not by the big publishers, not meant for ripping off the students, and not for free either. They are such as the books by Mike X Cohen, mentioned above, and he has made a series of books about Calculus, Linear Algebra, and Statistics, which all are great books at reasonable prices, and which work with Python or R (that you can use for free). I haven’t (yet) put all of of his books on the page, but they are all good and recommendable.
What is extra good about Mike’s books is that he has also made courses, which I think are available in different places, but I have seen some of them on Udemy – which makes them available at a low price.
The mix of course and book makes it easier for those of us who might struggle a bit with some of the details to get through it, and he’s both an excellent teacher and writer, so you really can’t go wrong with using his materials.
Some books available about mathematics and statistics are full of square root signs and greek letters, from the first page, and at times with hardly any text at all, while others have a focus on readable text – with illustrations and formulas, but with the text being the main aspect. I personally prefer the latter type, but people are different, so I have tried to pick some from each of these categories, or somewhere in between, so if you take a look at the list, please check out several books, because you might like one of them better than another.
The page for books and courses: Learning Resources
The page for products/software: Technical Resources
The page for datasets and tools: Data
Hopefully this all will be an inspiration to you. Should you have any comments or suggestions, please speak up. The comments section on this article is great for that, or if you want to message me privately, it can be done on Substack or through the contact form on the Inidox Data website.
Oh, and not to forget: I decided to make that separate site, because a lot of information is difficult to share the way I want to on Substack – it doesn’t allow much formatting of texts, or more advanced things like integrated courses or similar. Substack can do a lot, and the communicative aspect of it, with comments, chat, etc., is difficult to create yourself on your own website (it will be full of spam in no time), but it isn’t best at everything.