Mertz is now considered one of the finest guitarist-composers of the nineteenth century, his music having less affinity with the previous generation (Sor, Giuliani, Aguado) than with the piano music of Schumann and Mendelssohn. He is also often grouped with Giulio Regondi and Napoléon Coste, the other leading guitarist-composers of his time.

He learnt to play both guitar and flute to a very high standard, and had much success as a performer in Vienna, where he moved in 1840. When touring in Dresden, he met Josephine Plantin, whom he married in December 1842. When Mertz became ill in 1846, it is said that his wife accidentally administered too much of the strychnine that had been prescribed for him, making his condition worse. Nevertheless, he performed again in 1855 for Ludwig of Bavaria, but shortly after he had won the Brussels guitar composition contest in 1856, he died.

Mertz played a ten-string guitar, which was not uncommon at the time - Carulli and Makaroff also played such an instrument, Regondi and Legnani played eight-string guitars, and Coste played a seven-string. Mertz’s works are nevertheless playable on the modern six-string.