https://www.vikas.sh Vikas Paruchuri's blog https://www.vikas.sh/post/how-i-got-into-deep-learning I ran an education company, Dataquest, for 8 years. Last year, I got the itch to start building again. Deep learning was always interesting to me, but I knew very little about it. I set out to fix that problem. https://www.vikas.sh/post/how-i-got-into-deep-learning Thu, 11 Apr 2024 00:00:00 GMT https://www.vikas.sh/post/vision-for-ai-web I’ve been using the internet for more than 2 decades. Websites have evolved from static HTML and CSS to rich, interactive experiences. But the core navigation and usage of the web has stayed the same - we all mostly use Google to discover and browse information. https://www.vikas.sh/post/vision-for-ai-web Mon, 07 Aug 2023 00:00:00 GMT https://www.vikas.sh/post/semantic-search-guide Semantic search is a hot topic these days - companies are raising millions of dollars to build infrastructure and tools. I think due to this, most semantic search tutorials I see assume you need lots of tools like vector databases and LangChain. This couldn’t be further from the truth - for most use cases, you’ll be fine with just a few lines of Python code and no external dependencies. https://www.vikas.sh/post/semantic-search-guide Thu, 13 Jul 2023 00:00:00 GMT https://www.vikas.sh/post/startup-ambition It’s an axiom that startups are defined by growth. If it’s not growing 10% a month, then your company is seen as something less - a “lifestyle business”. https://www.vikas.sh/post/startup-ambition Sun, 31 Jan 2021 00:00:00 GMT https://www.vikas.sh/post/theres-no-one-right-way-to-build-a-business As I’ve scaled Dataquest, one of the hardest things for me to come to grips with has been that there is no one right way to build a business. This may be surprising to you. After all, it doesn’t seem like a very complicated truth. But the reasons why this has been a persistent belief for me are informative for others in my shoes. https://www.vikas.sh/post/theres-no-one-right-way-to-build-a-business Thu, 10 Dec 2020 00:00:00 GMT https://www.vikas.sh/post/how-to-rapidly-improve-your-management-skills It can be overwhelming when you start as a new manager, or when you’re an existing manager who is asked to take on more responsibility. https://www.vikas.sh/post/how-to-rapidly-improve-your-management-skills Sat, 04 Jan 2020 00:00:00 GMT https://www.vikas.sh/post/you-dont-need-to-perfect-to-be-a-good-manager The first year or two after transitioning into management can be incredibly difficult. Most new managers get promoted after they excel in an individual contributor role. But once you become a manager, it becomes clear that none of the skills and behaviors that helped you excel in your previous role will help you in your new one. https://www.vikas.sh/post/you-dont-need-to-perfect-to-be-a-good-manager Sun, 22 Dec 2019 00:00:00 GMT https://www.vikas.sh/post/i-barely-graduated-college I didn’t do very well in high school. My grade point average was around a 2.5 out of 4. I did well in some subjects that I was interested in, like math, computer science, and history, but everything else was a wash. The less homework a class required me to do, the better my grade in that class usually ended up being. In most classes I ended up counting down the minutes to them ending. I wasn’t particularly passionate about school, and I wasn’t one of those super driven high school students who always seem to be able to fit in homework, a social life, sports, and 10 clubs. https://www.vikas.sh/post/i-barely-graduated-college Thu, 12 Mar 2015 00:00:00 GMT https://www.vikas.sh/post/making-an-app For the past year, my girlfriend, Priya has been working on an awesome nonprofit called Tulalens. The idea is to be “yelp for low-income people in emerging markets”. She did a pilot in October/November 2014 where she and the Tulalens team went into the slums of Hyderabad, and surveyed pregnant women on which hospitals they went to and the quality of care they received. She was then able to analyze the data and figure out the best hospitals in the area. https://www.vikas.sh/post/making-an-app Thu, 19 Feb 2015 00:00:00 GMT https://www.vikas.sh/post/how-learning-to-code-kept-me-sane In January of 2011, I joined the US Foreign Service. Along with 80 others, I went through a class called A-100, and got a crash course on how to be a diplomat. We learned how to address foreign dignitaries. We got lessons in diplomatic history. There was even an optional class on how to comport yourself at diplomatic dinners (I skipped this one). At the end of training, we were ready to change the face of US foreign relations. https://www.vikas.sh/post/how-learning-to-code-kept-me-sane Mon, 29 Dec 2014 00:00:00 GMT https://www.vikas.sh/post/exploring-us-healthcare-data A few days ago, the Centers for Medicare and Medicaid Services (CMS) released some unprecedented data on the US healthcare system. The data consists of 9 million rows showing how much each doctor in the US charged Medicare, for what, and how much Medicare paid out. It doesn’t quite cover everything (for example, services with less than 11 beneficiaries were removed for privacy reasons), but its the best thing we’ve got. https://www.vikas.sh/post/exploring-us-healthcare-data Sun, 13 Apr 2014 00:00:00 GMT https://www.vikas.sh/post/simple-speech-recognition-in-python Sometime today, I got the idea to try to do automatic speech recognition. Speech recognition, even though it is widely used (and is on our phones), still seems kind of sci-fi-ish to me. The thought of running it on your own computer is still pretty exciting. https://www.vikas.sh/post/simple-speech-recognition-in-python Thu, 10 Apr 2014 00:00:00 GMT https://www.vikas.sh/post/an-easy-way-to-get-started-with-automatic-essay-scoring Wow, it’s been way too long since I have updated this blog! I am going to start making more frequent updates, and I have some cool things in the pipeline, so bear with me. https://www.vikas.sh/post/an-easy-way-to-get-started-with-automatic-essay-scoring Tue, 25 Mar 2014 00:00:00 GMT https://www.vikas.sh/post/what-makes-people-happy I’ve had a lot of different jobs over the past 4 years, and I’ve had some incredible experiences along the way. Lately, I’ve been struggling with what to do next. Or perhaps more accurately, I’ve been struggling with how to decide what to do next. Decisions that seem obvious in hindsight are tough to come to grips with beforehand, and it’s led me to think about what metric I am trying to maximize. I admit that it’s odd to think of life as a way to increase certain metrics, but aren’t we doing this already in a different way? A lot of people (myself included) will at some point say that all we care about is money. Isn’t that just us saying that money is the metric we want to maximize? Now that I am older and wiser (yeah, right), I find myself increasingly concerned with maximizing my own happiness. https://www.vikas.sh/post/what-makes-people-happy Thu, 14 Nov 2013 00:00:00 GMT https://www.vikas.sh/post/open-sourcing-movide I haven’t blogged in a while, mostly because I have been trying to figure out what I should do next. One thing that I have been working on lately that I am very passionate about is Movide. Movide is a student-centric learning platform. You might yawn at this point and wonder why Movide matters. It’s a natural reaction, given the crowded learning tools marketplace. Movide, matters, I think, because it is an open source attempt to change the LMS and learning tool paradigm. https://www.vikas.sh/post/open-sourcing-movide Sat, 19 Oct 2013 00:00:00 GMT https://www.vikas.sh/post/the-danger-and-power-of-visualizations I recently posted about visualizing the voting patterns of senators. In the post, I scraped voting data for each senator on every vote in the 113th Congress from the Senate website, and then assigned a code of 0 for a no vote on a particular issue, 1 for a yes vote, 2 for abstention, and 3 if the senator was not in office at the time of the vote (ie, a senator was switched mid-term). https://www.vikas.sh/post/the-danger-and-power-of-visualizations Wed, 07 Aug 2013 00:00:00 GMT https://www.vikas.sh/post/on-the-automated-scoring-of-essays We’ve all written essays, primarily while we were in school. The sometimes enjoyable process of researching the topic and composing the paper can take hours and hours of careful work. Given this, people react badly to the notion that their essays may be scored not by a human teacher, but by machine. https://www.vikas.sh/post/on-the-automated-scoring-of-essays Wed, 31 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/how-divided-is-the-senate I very seldom pay attention to politics directly, because politics have always seemed a bit circular and cyclical to me. Most of the political news that I take in ends up worming its way into the news sources that I do consume, like the excellent longform.org. Even given my limited intake of political news, one trend that I have noticed lately is the increasing number of references to the Senate as “polarized” or “divided.” Here is a link to an interesting series of charts on polarization. Is it possible to quantify this polarization? Can quantifying the polarization enable us to draw interesting conclusions? https://www.vikas.sh/post/how-divided-is-the-senate Tue, 30 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/making-instrumental-music-from-scratch I recently posted about automatically making music. The algorithm that I made pulled out interesting sequences of music from existing songs and remixed them. While this worked reasonably well, it also didn’t have full control over the basics of the music; it wasn’t actually specifying which instruments to use, or what notes to play. https://www.vikas.sh/post/making-instrumental-music-from-scratch Mon, 29 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/evolve-your-own-beats-automatically-generating-music Update: you can find the next post in this series here. https://www.vikas.sh/post/evolve-your-own-beats-automatically-generating-music Fri, 26 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/making-infographics-using-r-and-inkscape I have been making charts with R for almost as long as I have been using R, and with good reason: R is an amazing tool for filtering and visualizing data. With R, and particularly if we use the excellent ggplot2 library, we can go from raw data to compelling visualization in minutes. https://www.vikas.sh/post/making-infographics-using-r-and-inkscape Wed, 24 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/how-do-simpsons-characters-feel-about-each-other One day, while I was walking around Cambridge, I had a random thought – how do the characters on the Simpsons feel about each other? It doesn’t take long to figure out how Homer feels about Flanders (hint: he doesn’t always like him), or how Burns feels about everyone, but how does Marge feel about Bart? How does Flanders feel about Homer? I then realized that I work with algorithms – maybe I would be able to devise one to answer this question. After all, I did something similar with the Wikileaks cables. https://www.vikas.sh/post/how-do-simpsons-characters-feel-about-each-other Sun, 21 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/analyzing-audio-to-figure-out-which-simpsons-character-is-speaking Update: you can find the next post in this series here. https://www.vikas.sh/post/analyzing-audio-to-figure-out-which-simpsons-character-is-speaking Fri, 19 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/figuring-out-which-simpsons-character-is-speaking Update: you can find the next post in this series here. https://www.vikas.sh/post/figuring-out-which-simpsons-character-is-speaking Wed, 17 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/find-the-determinant-of-a-matrix The determinant of a matrix is a number associated with a square (nxn) matrix. The determinant can tell us if columns are linearly correlated, if a system has any nonzero solutions, and if a matrix is invertible. See the wikipedia entry for more details on this. https://www.vikas.sh/post/find-the-determinant-of-a-matrix Tue, 16 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/linear-regression-from-the-ground-up Linear regression is a very basic technique that we use a lot in machine learning. In a lot of cases (and I have been guilty of this), we just use it without much thought as to how the internals actually work. https://www.vikas.sh/post/linear-regression-from-the-ground-up Mon, 15 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/inverting-your-very-own-matrix I had my natural predilection towards math crushed out of me at some point in school, and after that point, Math (yes, we are referring to the higher power of math) and I had a wary understanding. I dabbled quietly, and Math turned a blind eye to me ignoring some of its deeper theory. When I stuggled loudly, Math did its best to hide its smirks. I generally refrained from throwing textbooks. https://www.vikas.sh/post/inverting-your-very-own-matrix Sun, 14 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/predicting-nfl-season-records-with-percept This is the first, non-technical, part of this series. See the second part for more detail. https://www.vikas.sh/post/predicting-nfl-season-records-with-percept Tue, 09 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/predicting-season-records-for-nfl-teams-part-2 This is the second, technical, part of this series. See the first part for the overview. https://www.vikas.sh/post/predicting-season-records-for-nfl-teams-part-2 Tue, 09 Jul 2013 00:00:00 GMT https://www.vikas.sh/post/my-talk-at-boston-python I just gave a talk at Boston Python about natural language processing in general, and edX ease and discern in specific. https://www.vikas.sh/post/my-talk-at-boston-python Wed, 26 Jun 2013 00:00:00 GMT https://www.vikas.sh/post/natural-language-processing-tutorial This will serve as an introduction to natural language processing. I adapted it from slides for a recent talk at Boston Python. https://www.vikas.sh/post/natural-language-processing-tutorial Wed, 26 Jun 2013 00:00:00 GMT https://www.vikas.sh/post/creating-a-wordpress-single-or-multisite-install-using-cloudformation-and-ansible I recently had to create some sites quickly. After evaluating a few options, setting up a wordpress multisite seemed like a good option. https://www.vikas.sh/post/creating-a-wordpress-single-or-multisite-install-using-cloudformation-and-ansible Wed, 19 Jun 2013 00:00:00 GMT https://www.vikas.sh/post/how-many-data-scientists-are-there I’ve seen a lot of articles lately about “Big Data” and the looming “talent gap.” This article from the Wall Street Journal is a good example. It cites a McKinsey estimate that states that we will need 1.5 million more managers and analysts who are conversant with “big data.” Of course, some of this is the media latching on the the next “big thing” (data), but some of it is true. Even anecdotal evidence, such as the number of job postings you find when you search for “data science,” indicates that there is a significant unmet demand for data analysis skills. https://www.vikas.sh/post/how-many-data-scientists-are-there Thu, 09 Aug 2012 00:00:00 GMT https://www.vikas.sh/post/tracking-us-sentiments-over-time-in Introduction https://www.vikas.sh/post/tracking-us-sentiments-over-time-in Mon, 18 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/finding-word-use-patterns-in-wikileaks 6/18: A follow-up to this post is now available [here](http://viksalgorithms.blogspot.com/2012/06/tracking-us-sentiments-over-time-in.html). https://www.vikas.sh/post/finding-word-use-patterns-in-wikileaks Tue, 12 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/nba-predictions-finals Now we are on to the finals! The algorithm enters the finals with a 6-4 record so far. Here is what we have for tonight:So, let’s see if OKC wins this one. https://www.vikas.sh/post/nba-predictions-finals Tue, 12 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/nba-playoffs-update-5-5-4 This is the sixth post in my series on [predicting the NBA playoffs](http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals-with-r.html) with an algorithm. After the Boston loss in their last game, the algorithm is now 5-4 in the playoffs. Hopefully it is correct tonight! https://www.vikas.sh/post/nba-playoffs-update-5-5-4 Sat, 09 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/nba-playoff-predictions-update-4-5-3 This is update 4 to my original post about predicting the [NBA playoffs with R](http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals-with-r.html). With the Thunder beating the Spurs and the Heat losing to the Celtics, the algorithm went 1-1 on predictions, making it 5-3 so far. https://www.vikas.sh/post/nba-playoff-predictions-update-4-5-3 Thu, 07 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/nba-playoff-predictions-update-3-4-2 This is my third update to my original post on [predicting the NBA playoffs with an algorithm.](http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals-with-r.html) Here are updates [1](http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff-games-results.html) and [2](http://viksalgorithms.blogspot.com/2012/06/nba-playoff-predictions-update-2-and.html). https://www.vikas.sh/post/nba-playoff-predictions-update-3-4-2 Tue, 05 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/nba-playoff-predictions-update-2-and This is my second follow-up to my previous two posts which were about [predicting NBA games with an algorithm](http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals-with-r.html), and my [first update to the algorithm](http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff-games-results.html). The algorithm’s record is now 3-1, as it correctly predicted Boston and Oklahoma City as winners of their past games. https://www.vikas.sh/post/nba-playoff-predictions-update-2-and Sun, 03 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/predicting-nba-playoff-games-results Game Results https://www.vikas.sh/post/predicting-nba-playoff-games-results Fri, 01 Jun 2012 00:00:00 GMT https://www.vikas.sh/post/predicting-nba-finals-with-r This is the initial post about the algorithm. See updates [1](http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff-games-results.html), [2](http://viksalgorithms.blogspot.com/2012/06/nba-playoff-predictions-update-2-and.html), and [3](http://viksalgorithms.blogspot.com/2012/06/nba-playoff-predictions-update-3-4-2.html) for more. The algorithm is currently 4-2 in the playoffs! https://www.vikas.sh/post/predicting-nba-finals-with-r Wed, 30 May 2012 00:00:00 GMT https://www.vikas.sh/post/loading-andor-installing-packages In R, the traditional way to load packages can sometimes lead to situations where several lines of code need to be written just to load packages. These lines can cause errors if the packages are not installed, and can also be hard to maintain, particularly during deployment. https://www.vikas.sh/post/loading-andor-installing-packages Tue, 08 May 2012 00:00:00 GMT https://www.vikas.sh/post/mapping-us-radiation-levels-in-r I have posted previously about the open data available on Socrata (https://opendata.socrata.com/), and I was looking at the site again today when I stumbled upon a listing of levels of various radioactive isotopes by US city and state. The data is available at https://opendata.socrata.com/Government/Sorted-RadNet-Laboratory-Analysis /w9fb-tgv6 . You will need to click export, and then download it as a csv. https://www.vikas.sh/post/mapping-us-radiation-levels-in-r Tue, 08 May 2012 00:00:00 GMT https://www.vikas.sh/post/monitoring-progress-inside-foreach-loop The foreach package for R is excellent, and allows for code to easily be run in parallel. One problem with foreach is that it creates new RScript instances for each iteration of the loop, which prevents status messages from being logged to the console output. This is particularly frustrating during long- running tasks, when we are often unsure how much longer we need to wait, or even if the code is doing what it is intended to. The solution to this can be found in the sink() function. This function redirects output to a file. I will show you a simple example of this using the iris data set. The code below will execute without printing any status messages, even though do.trace is enabled, which periodically displays the status of the randomForest. The random forest code is slightly adapted from one of the foreach package examples. https://www.vikas.sh/post/monitoring-progress-inside-foreach-loop Thu, 09 Feb 2012 00:00:00 GMT https://www.vikas.sh/post/using-latex-r-and-sweave-to-create LaTeX is a typesetting system that can easily be used to create reports and scientific articles, and has excellent formatting options for displaying code and mathematical formulas. Sweave is a package in base R that can execute R code embedded in LaTeX files and display the output. This can be used to generate reports and quickly fix errors when needed. https://www.vikas.sh/post/using-latex-r-and-sweave-to-create Tue, 31 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/parallel-r-model-prediction-building Modifying R code to run in parallel can lead to huge performance gains. Although a significant amount of code can easily be run in parallel, there are some learning techniques, such as the Support Vector Machine, that cannot be easily parallelized. However, there is an often overlooked way to speed up these and other models. It involves executing the code that generates predictions and other analytics in parallel, instead of executing the model building phase in parallel, which is sometimes impossible. I will show you how this can be done in this post. https://www.vikas.sh/post/parallel-r-model-prediction-building Fri, 27 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/analyzing-us-government-contract-awards As I was exploring open data sources, I came across USA spending. This site contains information on US government contract awards and other disbursements, such as grants and loans. In this post, we will look at data on contracts awarded in the state of Maryland in the fiscal year 2011, which is available by selecting “Maryland” as the state where the contract was received and awarded here. I will use Maryland as a proxy for the nation, as the data set for the whole nation will be a bit more unwieldy to analyze, and the USA spending site appears to need a significant amount of time to generate the data file for it. We may take a look at the data for the whole nation later on. https://www.vikas.sh/post/analyzing-us-government-contract-awards Tue, 24 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/r-regression-diagnostics-part-1 Linear regression can be a fast and powerful tool to model complex phenomena. However, it makes several assumptions about your data, and quickly breaks down when these assumptions, such as the assumption that a linear relationship exists between the predictors and the dependent variable, break down. In this post, I will introduce some diagnostics that you can perform to ensure that your regression does not violate these basic assumptions. To begin with, I highly suggest reading this articleon the major assumptions that linear regression is predicated on. https://www.vikas.sh/post/r-regression-diagnostics-part-1 Fri, 20 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/analyzing-federal-bailout-recipients-in I was searching for open data recently, and stumbled on Socrata. Socrata has a lot of interesting data sets, and while I was browsing around, I found a data set on federal bailout recipients. Here is the data set. However, data sets on Socrata are not always the most recent versions, so I followed a link to the data source at Propublica, where I was able to find a data set that was last updated on January 17, 2012. I downloaded the data in csv format. In the rest of this post, I will perform basic analysis on this data, and show that R can be used to do the same analysis as Excel in a much simpler and more powerful way. https://www.vikas.sh/post/analyzing-federal-bailout-recipients-in Thu, 19 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/intro-to-ensemble-learning-in-r Introduction https://www.vikas.sh/post/intro-to-ensemble-learning-in-r Thu, 19 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/build-your-own-bagging-function-in-r Bagging, aka bootstrap aggregation, is a relatively simple way to increase the power of a predictive statistical model by taking multiple random samples(with replacement) from your training data set, and using each of these samples to construct a separate model and separate predictions for your test set. These predictions are then averaged to create a, hopefully more accurate, final prediction value. https://www.vikas.sh/post/build-your-own-bagging-function-in-r Wed, 18 Jan 2012 00:00:00 GMT https://www.vikas.sh/post/time-based-arbitrage-opportunities-in I recently posted an [introduction](http://viksalgorithms.blogspot.com/2012/01/introduction-to-kaggle-algorithmic.html) to the Kaggle Algorithmic Trading Challenge, which I competed in. https://www.vikas.sh/post/time-based-arbitrage-opportunities-in Wed, 18 Jan 2012 00:00:00 GMT