A fine piece on him and his works by ST Joshi.
A provocative (?) quotation from the above:
His style, of course, has been much criticized, and there is no question but that his early work is "overwritten" in a way he himself later deprecated; but the later Lovecraft prose is as precise, musical, and evocative as anything out of Dunsany or Machen
In any case, Lovecraft is no longer (as of 2005) a guilty pleasure.
Much earlier, JL Borges went as far as to dedicate one of his short stories to him, although Borges didn't fully appreciate Lovecraft's style and genius (for that, there is Houellebecq, who pointed out that if there is something wrong with Lovecraft's style then obviously style does not matter).
P. S. Some other quotations from the Joshi piece I personally found interesting:
[I]n spite of his lifelong opposition to the extremist trends of modern literature – stream-of-consciousness, imagism, plodding realism – he could declare that Joyce's Ulysses and Cabell's Jurgen were "significant contributions to contemporary art"
In any case, his sonnet-cycle, while by no means radical, can take its place with the work of other conservative poets of the day – Rupert Brooke, Ralph Hodgson, Robert Hillyer, John Masefield, Walter de la Mare, and others.
Of Lovecraft's letters it is difficult to speak in short compass. In sheer quantity they dwarf the rest of his oeuvre to complete insignificance. Although at the moment they are known only to the inner circle of Lovecraft scholars, they are arguably some of the most remarkable literary documents of the century, and it is even conceivable that in the distant future his reputation will rest more on them than on his fiction. It is to the letters that we go for information on Lovecraft's life, for details about his literary work, for the particulars of his philosophical thought; but more than mere utilitarian adjuncts to scholarship, they are some of the most beautiful things of their kind. Lovecraft had no compunction writing letters of fifty, sixty, or even seventy pages; and it is in these heroic epistles – longer than most of his stories – that he reveals his true greatness and diversity as an artist.
Bonus: Lovecraft's useful essay on the history and then present of weird fiction including an embarrassing but lively misunderstanding of The Yellow Wallpaper.