I have been competing in various programming contests for the past 10 years. A lot, if you ask me. But, more importantly, I have competed in quite a few different kinds of contests. My adventure started with classic algorithms, and later I switched to optimization problems. Currently I compete mostly in machine learning contests (as a part-time job) and I also do some for-fun contests.
Considering that there aren’t too many people with such broad experience, I thought I’d write a (relatively) short summary of popular types of programming (algorithmic?) contests. This is not a complete list – instead I’m focusing on the ones that are most popular and (in my opinion) most useful.
The structure is type-oriented rather than site-oriented. This means that rather than flooding you with some random sites with an added description, I organized everything into categories and combined sites/contests with similar characteristics. Since some of the sites offer several types of competitions, I had to list them several times.
Ideally, if you’re confused or just curious about the whole world of competitive programming, this should give you a comprehensive overview of the subject.
Sometimes also called binary problems or (derogatorily) recreational algorithms. You’re provided with the problem statement and your goal is to write a, usually, short program that reads the input, processes it and outputs the computed result. Everything according to the given problem statement. In most cases, you have to submit the source code of your program, which in turn is compiled on a remote server and run against some hidden data. The thing common to all contests mentioned below is that your program is always considered either fully correct or incorrect, and there’s nothing in between.
Classic algorithms contests are usually an entry-level point for competitive programming. The difficulty range can span from things that are absolutely trivial to problems that only the problem writer is able to solve. They are a great practical exercise for many beginner programmers, as they give small, bite-sized challenges. At higher levels, they require highly-developed concentration, great long-term memory, problem-solving skills and often deeply-specialized knowledge. Getting decent at those contests develops a lot of skills that are easily-transferable to other areas in computer science. Especially if you focus on developing raw skills, instead of spending your time on solving hundreds of problems and hoping that you’ll see a similar problem in the future.
Both places are quite similar. Competitions are held a few times a week (links to upcoming contests: TC and CF). They last around 2 hours. Each format uses several (3 for TC, 5 for CF) original classic problems created specifically for that round (in general, this is always the case for every contest, but I wanted to make this clear). You submit only the source of the program and the program is executed remotely on the servers. Your solution, in order to be evaluated as correct, needs to produce correct output and run within specified time and memory constraints. The catch is that you learn that your solution is correct only at the end of the contest. When the round is finalized, you get your ranking updated (similar to ELO rating system) which quite accurately represents your current level. Whole user base is split into two different divisions (based on their current ranking) and each division gets a different problemset tailored towards their skill level. Also, after each contest, you can view submissions from other people, the editorial is published (which explains the correct solutions for the problems) and all of the problems are added to the practice area so that you can solve later the tasks that you were unable to solve during the contest. This makes them a perfect place for training.
Those were the similarities, so what’s different? The biggest downside of TC (apart from bad management, but that’s another story) is that it uses an archaic java applet for its competitions. Although that makes the first contest more complicated than it should be (the badly designed website doesn’t help), but in the long run it isn’t much worse than an “HTML5” UI. Another difference is that problems in each site have a slightly different focus. Tasks on TC are usually skewed towards problem solving, sometimes even resembling puzzles, while CF contains a lot of “filler” (another word for unimaginative) problems based on data structures, but this largely depends on the problem writer. Based on my experience, TC problems (on average) are slightly more interesting, but CF has more variety in the problems – mostly because they have more problems per round. Both competitions feature an opportunity to find bugs in other people’s codes (and get additional points if you’re successful at it), but CF designed it horribly and the best thing to do, is to ignore it entirely.
Annual Onsite Competitions
There are four big competitions: Google Code Jam, Facebook Hacker Cup, Yandex.Algorithm and TopCoder Open (algorithm track). All of them are quite similar. In order to qualify for them, you have to compete in a series of online qualifiers. Usually they are done in a knock-out fashion, reducing the number of competitors in each subsequent round. There are no restrictions on who can compete (well, unless you’re unlucky and your country is on the embargo list for USA). Rounds are short, between 90 minutes and 3 hours and feature only classic binary problems. If you manage to qualify, they will pay for your transportation and accommodation. They also give out prize money to winners, but for most people the most important thing you can win is either the trip itself, or landing a job in a top IT company.
The competitions have small differences between them (quality of problems, round structure, submission system), but there are two things that they all share: It’s really hard to qualify for any of them (it’s highly unlikely to qualify when you have less than 2 years of experience, even when you’re smart and dedicated). The other is that one guy won all of them in 2014. And considering that all of those contests have high variance, it’s an unbelievable feat.
It’s worth mentioning that nowadays, almost all contest sites allow some sort of practicing. So in theory, almost all sites serve as an online judge. The reason why I put online judges in a separate category is that they don’t offer anything else.
The are very few reasons why you should ever be on such site: You’re at the point where you feel you’re very bad at some type of problems and you’re searching for some very tough specific problems (which shouldn’t ever happen unless you’re aiming for being top100 in the world) and you can’t find them anywhere else. You’ve participated in a contest, and the problems were uploaded to a judge site so you want to try ones that you haven’t “AC-ed” or maybe some alternative solutions. You’ve got a lazy teacher for your algorithm class and he hates his job.
The standard example of this is a Travelling Salesman Problem. Those problems are characterized by the fact that you get a different score depending on the quality of your solution. And they are (or at least they should be) designed so that it’s impossible to obtain a perfect score.
Contests with optimization problems usually last much longer, and thus focus on different set of skills than their classic counterpart. Things like mental stamina, time management or out-of-the-box problem solving skills. In general, optimization problems require you to be much more versatile, but you don’t have to excel in any particular area. They are also closer to doing actual research, so if you want to tie your career with something other than software development, it’s a good idea to try them.
Unfortunately, there aren’t too many places where you can hone your skills in this area. Topcoder have Marathon Matches, but they no longer host regular competitions – something they would do until a few years ago. Most matches held under the MM banner fall into Machine Learning category (described below), the exception is the annual TopCoder Open tournament that has a Marathon category. Both the onsite final and the qualification matches use optimization problems. There are some rumors that they want to go back to doing regular matches, but so far nothing has changed.
Fortunately enough, while there aren’t too many contests, topcoder has a big archive of past problems used in those contests. So if your sole goal is to be better at optimization problems, you can practice on old problems. The upside of this is that you have an access to all of the winning solutions, plus usually there’s a “Post Your Approach” thread in the appropriate forum for each of those past contests.
There are also other contests where you can encounter optimization problems, but unfortunately I don’t think anything stands even remotely close to MMs. Probably the most popular contest in that category is Al Zimmermann’s Programming Contest, but the problems there are rather shallow and aren’t too interesting.
Sometimes mistakenly called data science. This is the place where the money is, because there’s a big demand for people who are specialized in ML. At least big compared to the demand for people specialized in classic and optimization problems.
ML contests require much more knowledge than the other types of contests, and in general they aren’t that entertaining. Still, they are currently the easiest way to get some hands-on experience in this field. If you need some motivation, remember that jobs requiring ML skills are some of the highest paid in the whole IT field.
Marathon Matches (again)
I could link to the same place I did above. Topcoder combined optimization and ML problems into a single category. Or to be precise, at some moment MMs category was expanded to include ML contests as well.
ML contests at topcoder come pretty much only in one form. Since whole topcoder is a crowdsourcing platform (plus community build around it), clients sometimes have problems that cannot be solved using “simple” software contests. In some of those cases, the problem they’re dealing with can be wrapped in an ML contest that features decent prize money for the best-performing solutions. Since ML contests are still MMs, the whole submission system works exactly the same way as in optimization problems. The only difference is that ML contests are usually not added to practice area.
Kaggle is a website that is pretty much build around ML contests. I’ll focus on differences between TC and Kaggle. The biggest difference is that in Kaggle you submit only the output of your solution instead of submitting the whole program and this has a lot of consequences. First of all, you get access to the entire data set and there are no limits on the languages/libraries you can use. There aren’t any time limits and you can even use a whole cluster to compute the result if you want (and can afford it). Since people don’t submit source codes, you can’t look at solutions at the end of contest. On the flip side, community is much more active and there are lots of “practice” contests where people share their solutions while the contests are still running. Contests on Kaggle also last much longer (usually 2 months for contests featuring a prize money).
Overall both sites serve a slightly different purpose and both of them have its pros and cons. Kaggle should be more friendly for people who are more interested in high-level knowledge rather than low-level inner mechanisms of how various ML techniques work, while TC is probably better for people with a strong algorithmic background. The other way of looking at it is that in Kaggle you (generally) use tools and in MMs you (generally) write your own.
24 hours of fun
15 years ago, the first Challenge24 was organized. I won’t go into the history of it, since I actually know very little about it. But I would describe this contest as 24-hour madness. It’s an annual team contest held in Budapest. You get around 15 problems. The scope of the problems is absolutely huge: classical, optimization, games over TCP/IP, computer vision, sound analysis, puzzles, and things that cannot really be described using words. You bring your own hardware for the contest and the whole competition is done offline. Honestly, this is the most entertaining contest I have ever attended. And I’m saying this, even though I had fever last time I participated.
Challenge24 inspired Deadline24, which in turn inspired Marathon24 (both held in Poland). They are somewhat simplified versions of CH24, but still a lot of fun. Instead of having lots of problems, they usually have 3 games and the rounds begin at the same time when the competition starts. For example, one of the past games was the classic asteroids game played simultaneously for 30 players.
Since there’s limited number of teams invited to the finals, each of those contests hold an online qualifier. Since those contests are way less popular than other annual competitions, it’s actually quite possible to qualify to one, without any significant background in algorithms (or even programming).
There is one additional annual contest worth mentioning that is somewhat similar to those 24-hours contests: Internet Problem Solving Contest. This contest has still a lot of tasks, although they have more resemblance towards the classic algorithms.
There are two big websites that were left out. The first one is CodeChef that has two different types of monthly competitions. Both of them are classic algorithms competitions. The other site is HackerRank, which is a mix of an online judge and a place which hosts some one-time contests that can cover all types of contests. The reason why I left those out is that both are well-known for the low quality of problem statements and in general you should avoid those (there are few exceptions though), considering there’s an abundance of other contests.
If you’re still in high school, then the most important contest for you is International Olympiad in Informatics. Each country has its own national olympiad with its own set of rules. In many countries, performing well at the national level of the IOI, is the easiest and safest way to get into the university of your choice.
ACM ICPC is a “classic algorithms” contest and it has the longest history of all of the contests/sites mentioned here. It’s an annual team contest for students. Each team consists of 3 people from a single university. The main goal of ICPC is the popularization of programming worldwide. This is achieved with the use of a complex multi-level qualification system and (also complex) eligibility criteria. This enforces diversity during the world finals. As a trade-off, depending on where you live, it’s either surprisingly easy to qualify for the world finals, or next to impossible. Practicing exclusively towards ICPC is probably the worst way to invest your time in programming contests as it promotes the most narrow skill set, the only exception is when you live in one of those “lucky” regions.
Honorable mention goes to Hello World Open for creating biggest bullshit competition of the year with a very successful marketing plan. If you look carefully, you’ll see that they even added their link in wikipedia page just after the launch of the contest. I guess if you produced the #1 top grossing app on mobile, you know that stuff pretty well. Seriously, topcoder should learn from them.
Another honorable mention goes to Imagine Cup. I attended Imagine Cup four times. Twice as a competitor, once as a judge and one last time as a helper. Over the years I was able to see how it devolves from an innovative annual competition that gathers students from many different areas (start-ups, digital art, algorithms/AI) to a shitty PR machine for Microsoft’s products. Judges were often selected due to political reasons and were in no way qualified to be performing their job. And all of the categories that didn’t directly promote MS stuff were discontinued. The truth is, Imagine Cups were the best onsite events I ever attended. At the same time, nowadays I would never ever recommend participating in them to anyone.
Project Euler is a special kind of an online judge (with a heavy math focus), where you use programming as a tool for solving problems, instead of it being the point in itself. If you really want to do some programming problems without the time pressure, I suggest you do Project Euler instead of any of the already-mentioned “OJs”. Also, since it’s actually quite popular, it will be much easier to find some help online in case you’re stuck on one of the problems.