Downscaling single magnetic bits to the ultimate size of individual atoms would open the possibility to maximize the magnetic storage density on a solid surface. However, realizing stable magnets of the size of one atom remained an elusive challenge for more than a decade. Recent advances show that single lanthanide atoms on suitable surfaces can preserve their magnetization on a time scale of days at a temperature of 1 K or below. Such properties enable the use of these atoms as stable magnets for low temperature experiments, opening a novel platform for testing magnetic recording techniques at the atomic scale. In this article, we describe the single atom magnets that have been investigated and the methods employed to address their magnetic properties. We will discuss the limitations of the present systems and techniques, and identify the challenges to close the gap towards potential future technological applications.