Where to Look for the World’s Best Compiler Optimizers:
A blog about some of the best optimizers in the world.
The world’s best optimizer–it’s a bold claim. I had a great time talking with Steve Blackburn at the 32nd International Conference on Machine Learning (ICML) in Helsinki, Finland, but I was a bit surprised when he told me that he thought Berkeley Research Analyzer (BRA) from UC Berkeley was the world’s best compiler optimizer. He then went on to give an excellent explanation of why he thought this, so I will defer to his expertise.
There are of course many other contenders for the title, but I have found BRA to be interesting because it is not often considered when we think about compilers and optimization. Therefore, some of my favorite discussions in terms of compiler design have been with Steve Blackburn and his team from UC Berkeley.
When thinking about compiler optimization in general, we tend to focus on GCC or LLVM because they are open source and production quality. However, there are many other commercial and academic projects that focus on optimizing compilers, so there is no dearth of options. But as Steve pointed out in our discussions, many of these optimizers target low-end
This blog post is a compilation of the best developer blogs that I read since 7/12/2017. I use Feedly to collect articles, and use tags to organize them. This list has been tagged with the following: **best compiler optimizers**, **machine learning*”, **artificial intelligence**, “**the future of the web**”. It takes me ~10 minutes to compile this list.
While a lot of the energy in the optimization world is focused on compilers, it’s not the only place to look for optimization opportunities. For example, there’s a lot of interesting work going on in machine learning and artificial intelligence (AI) to squeeze the most performance from GPUs and other processor architectures.
The two worlds are coming together nicely in this area of AI called “deep learning” that I mentioned in a previous post. According to many experts, deep learning will be one of the key enabling technologies behind self-driving cars, virtual assistants, speech recognition, and more. It also has broad applicability to scientific computing and data analytics.
As a case study for this post, I will focus on an optimization problem I recently worked on for Google Cloud Dataflow , which is an open source streaming data processing framework built on top of Apache Beam . I’ll talk about how we use various tools, including C++ AMP (and some intrinsics), cuDNN LAPACK routines and MPI ransidr routines, to achieve high performance with minimal code complexity.
What I’m going to do here is talk about advanced compiler optimization and how it’s done. This is not a tutorial, but I’ll try to explain some of the basic ideas as well as what they mean in practice. I’ll also try to give you some insight into how compilers are written and how they work.
Readers are expected to have at least one year of experience with C++, but this isn’t really a C++ blog, so I won’t assume any particular background. I will, however, assume that the reader has access to a computer or two and understands how to use them.
Let’s say you’ve written a compiler or an optimizer. You’ve spent many months (or years!) working on it, and you think it’s really good. But how do you know? How can you tell whether the optimizer is any good?
The usual way to test the performance of your optimizer is to compare the generated code against that of other compilers or other optimizers. It’s easy enough to do that by hand, but it’s also boring and time-consuming.
And if your optimizer is good enough, it will take a lot of effort for humans to match its performance.
That doesn’t mean you have no way to tell how good it is! In fact, there are lots of ways to check your code without generating or hand-editing code at all. Using these methods, it’s possible to know whether your optimizer works well in practice without having to write hand-optimized code at all.
In the last article we talked about how to go from C/C++ to assembly. In this article, I’ll cover compilers and assemblers for a few different architectures.
Sparc and Itanium both have free compilers (and assemblers) available for download. I developed some code in these languages in my early days, but honestly, it was kind of a nightmare. These are not beginner friendly tools and they don’t get updated much anymore.
If you are building a new kernel or device driver, you might need to write your own assembler or compiler. If this is the case, you will have a lot of work to do no matter which architecture you choose. I’m going to list several architectures and leave it up to you whether or not you want to research them further. The only reason I’m listing them is so that I can tell you about some of the optimizations that are available if you do decide to go with one of these architectures:
IBM/Motorola PowerPC has a freely downloadable compiler called GCC which supports this architecture. It actually works quite well and has been around for many years. There is an optimization package available from IBM called “Blue Lagoon” that is supposed to be very good at optimizing code for Power
In the past few years, there has been an explosion in the number of data scientists, data engineers, and data analysts working in fields such as finance, retail, insurance, advertising, manufacturing and government. Now that Big Data is becoming a household term it seems like everyone is hiring for analytics teams. And with good reason. As the world becomes ever more digitized we are collecting more and more data about customer behavior and business processes.
Tens of thousands of people have been hired over the past five years to work on analytics projects in every industry you can imagine. But what do these people really do all day? And how do you get a job doing this?
I wanted to learn more about this field so I reached out to some people who are working at the forefront of this movement. These are the people responsible for building out analytics teams at companies like HP, eBay and Twitter. They are also leading the open source community around predictive modeling and advanced analytics software development such as Julia Computing, RStudio and Revolution Analytics.