Day 1. Our intro-to-data science lecturer is talking but I can’t pay attention because I hear beeping in another room and it’s driving me nuts. I get up to inspect, expecting to find a 90’s-era wristwatch or a Tamagotchi in its final death throes. Instead I see what looks like the lovechild of a Mars Rover and a Roomba vacuum cleaner, beeping its way across the floors, gobbling up cigarette butts. Its creators, members of the previous batch, explain that this was their three-month project — and that, more importantly, yes they can turn the beeping off.
I return to my introductory class feeling about as relieved about the noise as I was intimidated by the skills differential. Was I in too deep?
What’s my technical background?
I studied law, graduating in 2009. While interning at a London law firm, I realized that I’d rather be the one creating legal predicaments than the one cleaning up the mess afterward. That left two options — criminality or entrepreneurship. After much soul-searching, I settled for entrepreneurship. So, I started self-studying web development, putting entire programming languages onto flash-cards, back in my youthful, occasionally RSI-filled zeal. I combined law and code to create a micro web-business selling law outlines, a business that has (mostly) managed to pay my keep ever since.
So while I’m experienced as a programmer, I’m weaker with math, having not taken a quantitative degree (the only quantification lawyers do is counting billable hours). I attempted to catch up by studying some linear algebra, calculus, and probability before starting bootcamp, but — if I’m honest — the math stuff is much shakier.
What motivated me to study data science?
At one level, I have mercenary motivations, as I can see ways to profitably apply machine learning to my business (e.g. I want to predict good offers to send over email plus I want to improve my SEO by extracting case law and statutes from the documents I sell.)
Perhaps an even bigger motivation is that I like learning new things. Taking time out to focus on studying is a treat for me in the same way that a long holiday might be to someone less geeky and with nicer hair.
Lastly, as a web developer, I’ve been feeling serious FOMO about machine learning. I wanted to get up to speed with the latest tech of our times and quell the fear about getting left behind.
Why did I choose a bootcamp over teaching myself?
This was a really tough decision for me, since, like many self-taught programmers, I pride myself on being an autodidact. I once even wrote a 40-page piece on the topic autodidactism... so you can imagine I had more than a little cognitive dissonance about signing for camp. Why would I need an external institution to teach me data science when I learned web development just fine on my own?
What swayed me were two pragmatic considerations:
- Web programming is more conducive to learning on one’s own than data science. With web development, the feedback loop between writing code and testing it out is tiny. You can try ten variations in as many seconds. Often your mistakes are obvious and visible — say if the wrong data appears on a web-page. But the same isn’t true with data science. The feedback loops are painfully slow. Some models take hours or even days to train, so you don’t get the luxury of blindly hitting “up and enter” until it works. What’s more, mistakes in data science are more difficult to diagnose. Typically the output is a single number describing how good or bad your predictions were. When you get a rubbish result, almost anything could be the cause — and often it’s something as subtle as an erroneous assumption in your thinking about the model. Having expert instructors on hand would, I hoped, allow me to avoid these wasteful time sinks.
- Even though I taught myself to code, I noticed that I had the fastest rate of learning when working on projects with other programmers in the room, soaking up all their real-world techniques and productivity-enhancing practices. Now that I’m in my 30s and have more responsibilities, I place an even higher premium on learning quickly. But I don’t have a network of data scientist friends to do projects with, never mind ones with that kind of free time. Therefore a bootcamp seemed like a reasonable way to embed myself in such an environment.
How did I choose between bootcamps?
For what it’s worth, I ended up in Data Science Retreat, studying in their Berlin office. (The same team runs a parallel bootcamp called AI Deep Dive in Toronto.) I have no idea whether my bootcamp is any better or worse than the many other ones out there — my sample size is only one! But what I can do is share my experiences in a way I hope generalizes to anyone comparing options.
- A good match between your programming skill and that of your cohorts: Programming is the sine qua non of data science. Without basic programming competency, even the greatest statistician won’t get anything done. But programming experience between students differs wildly. After all, IT is a field where the best practitioners are often thought to be 10x more productive than even average ones. As such, choose a bootcamp where candidates are required to have a level of programming skill that somewhat matches your own. Otherwise, you’re going to feel frustrated when your deep-learning class becomes a lesson in how to call a python function with more than one argument.
- Pacing: Relatedly, choose a course with pacing that suits your style of learning. When I met with attendees of another bootcamp, we came up with a metric to calibrate the relative pacing of our programs: $time_spent_learning_pandas (pandas is a python library for viewing, cleaning, and fixing data). Ours was two days to their five. Neither is “better”, it just depends on preferences about how quickly you’d like to cover material and about whether you prefer to go deep or go wide.
- Diversity: Some bootcamps seemed to have classes that were exclusively white and male. If that doesn’t sound like your cup of tea, check to what extent the bootcamp you’re considering markets to and includes people from other demographic groups. In mine at least, three of the ten students are female, as well as two of the teachers (so far).
- Class size: Obviously, the less the better. Be sure to distinguish between initial class size and actual class size two-weeks-in. Although we started off with ten in our batch, two people have since dropped out completely (one because the course was too difficult and another for personal reasons). Additionally, an average of two people miss each teaching session, bringing our actual class size down to a much more intimate six heads.
- Do the teachers work in industry or are they full-time pedagogues? Generally I preferred teachers from industry. It was motivating to hear how real-world companies do things, since often they used surprisingly standard techniques and achieved satisfactory results. What’s more, instructors from industry can help you get hired afterwards. One instructor told us that he studied at this same bootcamp four years ago and got his last three jobs via his old tutors. That said, there is a downside to having instructors from industry: Because they already have day jobs, they might only be able to teach on the weekend or on state holidays. The consequence is that your schedule will become irregular. Which brings me to my next point…
- Schedule: Our schedule was more erratic than I expected, with a lot of tuition happening on weekends. The bootcamp operators more-or-less expected students to give themselves completely to their studies during the 3 months. The intensity is rewarding but exhausting. I mostly managed to adjust, but people under tighter time constraints should probably enquire about their bootcamp’s schedule beforehand to ensure they won’t clash.
- In town or out of town? I chose to study in Berlin, where I’ve
lived for the last seven years. However, most of my cohort are visiting from elsewhere. I’m jealous: I think there’s a big advantage to studying away from home, in that there are fewer distractions, fewer social obligations to attend to. One guy in my batch didn’t tell any of his Berlin friends he was in town, so as to better concentrate on the material.
- How much clout do the teachers have in the data science community? All things equal, I’d prefer bootcamps that have tutors with strong public reputations (popular articles, GitHub repos that are actually used by real data scientists, etc.) This provides some assurance that I’m being taught by people who legitimately know their stuff. I was delighted when I saw that an author I’d already read on Towards Data Science tutored us for three days.
- Does the bootcamp have a local community? There’s a saying out there that the best teacher is someone who is just one step ahead of you, since this person will still identify with the difficulties you are currently going through. My bootcamp had a spare room in its office that was offered as a makeshift coworking space for previous attendees of the bootcamp. This offer was often availed of, meaning there were usually a few veterans wandering around, happy to flaunt their newfound knowledge and answer my basic questions. Another form of community I noticed in bootcamps was the hosting of meetup.com events in their offices after hours. Such events, when they occur right on your doorstep, would be a great way to network with practitioners and squeeze in even more data science.
What did I like about bootcamp?
- My absolute highlight so far was completing my first end-to-end project. The bootcamp split us into teams in which we would spend three days competing in a Kaggle challenge to predict sales for a pharmacy. Even though we’d already had classes on various tidbits, I had no idea how to stitch everything together into a real project that spits out predictions. Spending three days doing the challenge was extremely effective for consolidating what I’d learned. The instructors, aside from
floating around to help us get unstuck, also competed. Obviously they beat the student teams roundly, but I did feel a sense of achievement seeing that my average error was only a few percent off of theirs. This made me realize that you can get passable results with just the basics. If I applied a similar model in my own business, it would already be valuable! I also learned a ton by comparing the instructors’ solution to my own. (My team’s
code is here in case you are curious.)
- I’m motivated by competition. So when the instructors give us exercises and I’d notice that someone else was three questions ahead of me, I would feel a strong urge to catch up. This would prompt me to study harder after classes — more than I would otherwise.
- Somewhat relatedly, in topics where I was more knowledgeable than my peers, it felt rewarding to assist them.
- Socially, I’ve enjoyed getting to know people outside my usual social circles. For example, one student is a Kuwaiti banker/entrepreneur who tried to do Netflix for video games in the days before streaming. Another is a neurobiology researcher interested in cognition.
- I appreciated the tutors’ help in getting a clean development environment set up. The whole internet seems filled with advice about installing anaconda and using these godawful notebooks. I had been led to believe that the world as we know it would end if I didn’t follow this advice to a “t”. Not so! The instructors showed me how to run everything smoothly with a regular python virtual env and my trusty old VIM editor of ten years.
What did I dislike about bootcamp?
- Some people (likely including me) are greedy with questions during tuition time and ask about things that have zero general usefulness to the class. But these questions have a cost, in that they force the instructors to cut out core material that would otherwise have been covered.
- Some people (again, sometimes including me) turn up late, and it’s frustrating to wait there twiddling your thumbs while the tutors provide a grace period for the tardy, with the side-effect that it reduces everyone else’s tuition time. I would prefer if the classes started punctually — with or without full attendance. Indeed, when tutors provide grace periods it encourages even more future lateness.
- Downloading and installing packages for machine learning libraries can take north of 40 minutes. Our instructors sometimes asked us to install a bunch of stuff the night before, but if some students missed the memo or just didn’t bother, it cut into teaching time.
- It’s easy to get dangerously over-confident and land yourself in embarrassment. One time I got cocky about my code being correct — and, more foolishly, about one of the instructor’s code being wrong. So much so that I confronted them publicly about it. Eventually they looked at my
screen and it took them all of about two seconds to locate my (incredibly sloppy) error. They handled it gracefully, but I felt angry at myself and more than a bit humbled to be back in the “dumb beginner” stage. In short, bootcamp is hard on the ego.
- There can be variance in instructor quality. Although I liked most of our teachers, there was one whose dismissive tone caused me to become scared to speak up in the class. I don’t think he was malicious per se — more like poorly socially attuned. Although I would have preferred if my bootcamp never hired him, they did ask for my feedback on his class the next day and were sorry I had a bad experience. The true test of will be whether the operators drops this tutor for the next batch…
What skills have I improved after three weeks?
This bit is going to be a bit heavy in jargon so feel free to skip.
- I developed some intuition for how the major machine models work via 3-days of interactive exercises playing with simplified versions of the algorithms. For example, here’s a video showing how changing the bias and the slope of a regression model affects the total error amount (SSE) and the distribution of error sizes.
- I started adopting the paradigm of vectorized calculations, i.e. where functions are called on vectors (or matrices) instead of on normal lists. Vectorized calculations can be 100x faster than regular Python loops, e.g. because the processor doesn’t have to worry about the possibility of subsequent items in list-like structures being of different data types (and therefore needing alternative machine instructions to be swapped in).
- I can now explore datasets with pandas — e.g. view column types, find nulls, calculate means for subgroups, locate outliers, and filter rows:
- I can replicate many SQL queries with pandas. For example:
- I can predict memory usage of matrices used in machine learning. It’s actually stupidly easy — if there are 60k rows and 10k columns in some output matrix of 8-byte floats, that means it will require 600 million slots, each of size 8 bytes — i.e. 4.8GB of memory.
- I can graph data during exploratory analysis — e.g. to compare the means on different days of the week for sales.
- I realized the value of using assertions all over my python code so as to stop errors creeping in when I change “just one thing”. E.g.
- I know how to build and structure a small data science project — how to split data to test/train, how to graph during exploration, what my options are for cleaning up problematic records, how to feature-engineer in a clean way, how to train a pre-packaged model from scikit-learn, how to test the accuracy of a model, how to using automated tools to search for better model parameters (e.g. GridSearch), how to save a fitted model to disk so it doesn’t have to be trained from scratch again, and, finally, how to package everything up as a shippable program that makes predictions on unseen data.
- I am now aware that although the data science field is huge, there are a few go-to algorithms that perform well in many situations and probably do a good enough job for “average” data science problems at “average” companies. These models include random forests, logistic regression, linear regression, K-nearest neighbors, dimensionality reduction, and, of course, neural networks.
- I know how to speed up the feedback loop when developing a data science model. E.g. Once my training data is cleaned, I now save it as an intermediary CSV file so that the (often slow) clean-up computation does not need to be carried out again and again. And I give my code a “debug mode” where the model to be fitted is given parameters that make it train almost instantly: This gives me a way to sanity-check the overall flow of the code without waiting forever.
- I finally got my head around gradients in multi-variate calculus. This was thanks to a combination of excellent videos and websites recommended by tutors and to two of my classmates who patiently went through my pen-and-paper derivations and de-muddled my thinking.
- (On my own) I learned how to combine scrapy, the lovely python tool for scraping data from websites with Scraping Hub, a service that makes it really easy to run scrapers in the cloud.
- (On my own, again) I optimized my editor (VIM) for python. I enforce a nice style with autopep8, I reformat long lines automatically with yapf, I detect syntax errors with pyflakes, I get auto-complete and function documentation with jedi-vim, and optional typing hinting using mypy. And, last but not least, a shortcut for inserting a debugger snippet or printing the current line.
How would I advise someone to get the most out of their bootcamp experience?
- Be reasonably proficient with programming before starting: If you can’t code, you’ll be reduced to working at a snail’s pace. Honestly I think it’s a waste of your time and money to attend a data science bootcamp if you haven’t already spent at least a hundred hours doing regular programming first.
- Be willing to ask questions to instructors: Some students are too embarrassed to ask questions even in private (after class) and this means they are missing out on instructor expertise, one of the key advantages of being at a bootcamp. Also, there’s a danger of getting caught in a rubbish Nash equilibrium where nobody understands what’s being taught but everyone thinks everyone else understands because no-one else is asking any questions…
- Ask other students for help: Our batch had people with various strengths — whereas I have a lot of experience with classic programming, others were pros at solving difficult equations using computers. Tapping into your peers’ knowledge is a great way to get unstuck. I also learned to be more attuned to the indirect, face-saving ways that other students might ask for assistance — e.g. someone asked me “Have you modified the code to do XYZ yet?”) I didn’t realize until a day later that their real meaning was “Can you share how you did XYZ with me because I’m lost.” Compare this to another someone more upfront who might just say “Can you send me your code for XYZ?”
- Play to the intellectual interests of the instructors: If an instructor’s passion is time series data or bayesian stats, they are prone to talking your ear off about these topics. That means you could be strategic in your choice of projects so as to tap into their excitement.
Getting a job afterward?
Since I already own my own business, this isn’t a consideration for me. But for many others on the course, getting a job afterward is a major motivation so I’ll add a few observations.
I met quite a few people from the previous batches of my bootcamp and heard the whispers about how they fared on the job market: The results turned out to be totally unsurprising. Basically anyone who was driven, who could ship working code, and who had a bit of social grace was snapped up by industry. I say this is “unsurprising” because the exact same people would probably have been snapped up without having done any bootcamp. But probably only for a regular programming job, not a data science one.
I realize my point is almost a tautology: if you want to be hired as a data scientist after bootcamp, be the kind of person who was hire-able anyway.
Conclusion: Was I in too deep?
On my first day, I felt terrified that I was under-qualified after seeing how a team from the previous cohort had built a freaking AI robot. But I shouldn’t have been so disheartened. The thing is: programmers build upon the work of other programmers. In this team’s case, they’d based their work on Donkey Car, an open-source DIY self-driving platform for small scale vehicles.
I don’t say this to belittle that team’s achievement — what they’d done was truly impressive. I say this to demonstrate how today’s tools allow even beginners to perform feats that seem indistinguishable from magic to other programmers — let alone to the public at large. This is exactly what’s so exciting about data science.
I’m no longer feeling scared — I’m feeling inspired.