The sublime is an aesthetic -- a theory of the way we perceive and are affected by certain phenomena, particularly in cultural representations or interpretations -- that has its origins in ancient Greece. A work long attributed to the Greek philosopher Longinus [pronounced something like "lawn-JINE-us"] theorized about the way certain aspects of rhetoric affect those who listen to powerful speeches. In the Eighteenth Century, the idea of the sublime was resurrected and extended; it became used as a way of talking about the impact on us of certain dramatic or powerful manifestations of nature (towering mountains, storms, avalanches, etc) or supernature (demons, angels, ghosts). My personal suspicion is that debate about the sublime (and the beautiful and the picturesque) became important in the C18 because writers and philosophers were increasingly interested in how our minds work, and in the days before psychology as a science, these discussions of aesthetics were another way of talking about the operation of our minds.
In the 1750s Edmund Burke published a book on aesthetics, A Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful, that became the most influential contribution to this ongoing aesthetic debate. Linking the sublime, and its concern with powerfully affecting phenomena, to terror, Burke effectively created a sort of "how-to" book for writers interested in cultivating powerful emotional responses -- including "negative" responses such as fear, dread, suspense, and horror -- in their readers. It is no accident that the most influential Gothic novelist, Ann Radcliffe, was significantly influenced by Burke's theories about what makes a thing sublime and how that thing affects us, psychologically. (Radcliffe handled Burke's aesthetic masterfully; many less talented writers used Burke's book as a catalogue of scene-setting, more or less running through the list of things that Burke identified as agents of sublimity, packing as many as they could into their novels.)
A number of aestheticians and writers important in the development of the sublime are treated on separate author pages here at The Literary Gothic:
For more on the aesthetic of the sublime, and its role in shaping both the atmospherics and the psychology of Gothic literature, see the following:
A very helpful overview of the sublime and its impact on literature. [Monika Fludernik, U Freiburg; Literary Encyclopedia]
The Sublime: An Overview
Longinus' On the Sublime
The book that started it all -- even if it's about the rhetorical sublime, and probably not really by Longinus....
Edmund Burke on the Sublime
Series of brief extracts illustrating Burke's theory of the sublime. [W. W. Norton]
"Of the Sublime" 
By Friedrich Schiller, amplifying some of Immanuel Kant's ideas about the sublime.
"On the Sublime"
Another work by Schiller examining the sublime; first published in 1801. [Schiller Institute]
"The Sublime and the Domestic: Structures of Mediation in the Gothic
A helpful mini-discussion.
Incorporating the Sublime: Burke, Kant, and Freud on the Sublime
Brief excerpts illustrating the development and use of the sublime in Gothic fiction.
The Kantian Sublime
Paul Crowther's important book on the sublime and Immanuel Kant; access to the full text is for subscribers only, but abstracts of each chapter are free.
An overview based on thumbnail sketches of the theories of Longinus, Burke, and Wordsworth. [Janice E. Patten, San Jose State U]