Properly placing stones in a structure is the key to the art of masonry. Stones have a natural bed on which they want to be laid; this natural bed is the direction in which they were pulled out of the ground in the quarry. How the stones are placed can have significant ramifications to the life of a structure.
Natural Bed
The natural bed of the stone is the direction in which came out of the ground. For sedimentary and some metamorphic rocks, this natural bed is usually evidenced by a subtle banding showing how the grains of the stone were originally deposited. This banding should be horizontal if the stone is sitting in the position it was originally deposited.

Stones should almost always be placed on the natural beds, with the pressure in the same direction as it was before the stone was quarried. Why? Because they are vastly stronger in this direction, having sat in this direction for untold years, being compressed together. And, for sedimentary rocks, the pressure holds the layers together. Place a stone on edge with high pressure, and the stone can delaminate like leaves of a book!
Applying Pressure
So the layers of a stone should be placed with the pressure holding them together. This is especially true for arches, as the pressure in an arch is relatively high, and durability important. Occasionally in a wall, a stone might be placed on end rather than on the natural bed. Usually, the main reason for this is to fill a very narrow gap. If this is done, the next stone above should generally not be allowed to rest on the stone placed on end; rather it should bridge the gap. This is to reduce the pressure that can delaminate the stone placed on edge.
Boulders
Hillside boulders, even if sedimentary, will often be found to have no clearly defined natural bed, at least at a causal glance. If a close look at them shows how the grains were deposited, the stone should be placed such that the weight is applied at right angles to the grain of the stone. It will be found that generally boulders are more forgiving to being placed on end, as they have weathered differently over the years from quarried stone, and are much harder and more cohesive.
Conclusion
In the end, especially for stones bearing a substantial weight, it is of the utmost importance to place them on the natural bed to ensure the long-term durability of a structure. Arches in particular should be built with care so that the pressure is pressing the natural bedding together. (This means, by the way, that the keystones will actually have the bedding lines vertical, as the line of pressure at the keystone is applied horizontally from the ends.)
Even in walls, stones placed on end should be avoided, with the possible exception of small rocks being used to infill a narrow gap between larger stones. Any rocks placed on end should not be subjected to great pressures. Finally, the more amorphous the rock is in nature, the better it can handle forces placed against the natural bed.