The cost of reservations depends on the country and the specific train you choose to take.  Most trains don’t require reservations, but the fastest, most popular, trains between major cities usually do.  Most reservations cost about 5 euros, but some are as expensive as 19 euros.  I think the best list is at www.railpass.com  Click on “info center” then “using your railpass” then “supplements” for a list, by country, of the trains that require reservations or supplements and the approximate cost to buy them at the train station in Europe.

You can buy reservations for almost any train in Europe at most major train stations, so you do not have to buy them at the station you will be departing from.  If you have a set itinerary, you can buy all the reservations you need at once.  Another time-saver is to by the reservation for the train you’re going to take when you leave a city upon arrival in that city  —  before you leave the train station to save an extra trip to the station. 

can I just show up like an hour before we want to hop on a train and expect a spot? Or is it better to just suck it up and pay for the reservation?

Definitely make reservations! You probably won’t get a spot on a train if you show up an hour before, unless it is a quick train like Nice to Monaco. Oldlady mentions that they can be as high as 19 euros and I think most are closer to that than 5 euros for a reservation. I don’t think I ever paid as little as 5 euros for a resrvation and I was just in Europe a month ago.

If you choose a train that requires a reservation, then you must have a reservation to board the train.