inspect the check valve too.
start the car and put your foot on the brake. you should have power assist at this time. keep your foot on the brake and shut off the car. if your pedal gets hard and rises up, replace the valve first. it is a one way valve which is supposed to hold enough pressure for a good 2 more brake applications when there is no vacuum. if, after replacing this ~$5 part, it still does it, you may have a bad booster. if the booster is leaking internally, you should be able to hear it hissing.
your engine also has to be supplying a good amount of vacuum for the booster to work. so in addition to checking for other vacuum leaks, if you have a major thumpity thump camshaft, you may need to get a vacuum pump, aux. canister, or use hydroboost instead.