If you have iTunes set to manage your music/copy files to your library (which you really should), you'll have an iTunes folder. On a Mac, it would be at ~/Music/iTunes. (~ means "your home directory").
That iTunes folder contains several files, such as iTunes Library Extras.itdb, iTunes Library Genius.itdb, iTunes Library.itl, iTunes Library.xml, and at least two folders, Album Artwork, and iTunes Media. Inside iTunes Media are several further subfolders.
Anyway, the iTunes folder, despite how you can look inside of it, is best to treat as a single unit. You never change stuff inside it manually (though you can put music into the "Automatically Add to iTunes" subfolder). If you're adding music to iTunes from some random folder, it should make a new copy of the music in the iTunes library, and then you should delete the original files. And you never keep your music elsewhere, even though the iTunes settings give you the option to do so. You can simply move the whole unit from place to place. In fact, if you keep it on an external drive, you can open it on one computer, then another. (Even between Mac & PC though that's not recommended for technical reasons I won't get into.)
One quick point: When you add music to iTunes from *within* the iTunes folder structure, it doesn't make a new copy; it simply moves the music to a folder that is determined by the file's metadata. This is what happens with the "Automatically Add" folder, also, if you have music in your Music subfolder that for some reason is not reflected in your library, you can just add it and it won't duplicate. In fact, you you add your entire iTunes Music folder, that is in your iTunes library structure already, back into iTunes, it won't actually duplicate anything you already have. Or at least, not the last time I checked.
When you launch iTunes, holding down option lets you choose to open one iTunes library or another. It's always going to pick the last library it opened by default. You can open different libraries in iTunes in the same sense you can open different documents in Word.
When people lose their libraries, sometimes what has happened is they have a library on an external drive, and they launch iTunes when the drive is not connected. So iTunes creates a new, empty library in the default location, and opens that every time, until you tell it otherwise, by holding down "option" upon launch. (Shift, I think, on a PC.)
Anyway, the above might not be at all your issue. Your hard drive may have flaked out, etc. But understanding exactly how iTunes deals with libraries & the location of files would save a lot of people a lot of trouble.