Best Machine Learning Research

In this article, I’ll help save you some time by curating the current large pool of research efforts on arXiv.org down to the manageable short-list of my favorites that follows: here’s the best machine learning of 2019. Enjoy!

Tackling Climate Change with Machine Learning

Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help. This paper describes how machine learning can be a powerful tool in reducing greenhouse gas emissions and helping society adapt to a changing climate. From smart grids to disaster management, the authors identify high impact problems where existing gaps can be filled by machine learning, in collaboration with other fields. Recommendations encompass exciting research questions as well as promising business opportunities. The researchers call on the machine learning community to join the global effort against climate change.

 

Reconciling Modern Machine Learning Practice and the Bias-variance Trade-off

Breakthroughs in machine learning are rapidly changing science and society, yet our fundamental understanding of this technology has lagged far behind. Indeed, one of the central tenets of the field, the bias-variance trade-off, appears to be at odds with the observed behavior of methods used in the modern machine learning practice. The bias-variance trade-off implies that a model should balance under-fitting and over-fitting: rich enough to express underlying structure in data, simple enough to avoid fitting spurious patterns. However, in the modern practice, very rich models such as neural networks are trained to exactly fit (i.e., interpolate) the data. Classically, such models would be considered over-fit, and yet they often obtain high accuracy on test data. This apparent contradiction has raised questions about the mathematical foundations of machine learning and their relevance to practitioners. This paper reconciles the classical understanding and the modern practice within a unified performance curve. This “double descent” curve subsumes the textbook U-shaped bias-variance trade-off curve by showing how increasing model capacity beyond the point of interpolation results in improved performance. The paper provides evidence for the existence and ubiquity of double descent for a wide spectrum of models and datasets, and it posits a mechanism for its emergence. This connection between the performance and the structure of machine learning models delineates the limits of classical analyses, and has implications for both the theory and practice of machine learning.