The question of what size should my synthetic stay be to replace the metal stay comes up a lot. As always, there are two methods to figure this one out.
The first is to calculate your RM30. RM30 is the force that is required to heel the boat over 30 degrees. There are many factors that play into this number, but they will give you a good idea of the loads your shrouds will experience while sailing heeled over at 30 degrees. Once again, there are two ways to calculate this value, one is via an actual test performed on the boat while in the water, the other via mathematical equations.
With this number in hand, you can safely calculate the size of your standing rigging knowing the loads that it will be subjected to.
The other method is to base it off of the standing rigging that the boat was originally designed to have. Steel standing rigging is sized so that the maximum amount of tension applied to it is 20% of its breaking strength. While your standing rigging should never be set this tight, this is the safety margin in steel rigging.
If you have 1/4 inch 1x19 316 stainless steel standing rigging and wish to know what size your synthetic stays should be, simply do some simple calculations.
1/4 inch 1x19 has a breaking strength of around 7600 pounds.
20% of 7600 is 1520 pounds
Synthetic standing rigging is sized based on creep rather than breaking strength. Synthetic standing rigging will creep less if it is under less static load. Keeping the static load below 15% will keep creep down. If the load is less than 10% of the total strength of the dyneema, creep will be significantly less.
With our example of 1/4 inch 1x19 SS wire with a 20% load of 1520 pounds, we can safely assume that using 6mm New England Ropes STS-HSR with a breaking strength of 12,400 pounds would be a safe choice. 1520 pounds of static load would be 12.3% of its total strength, keeping the creep to a safe amount. Sizing up to the next size would reduce creep considerably but also increase windage.
7mm New England Ropes STS-HSR with its breaking strength of 18,700 would be loaded at a mere 8.1%. Creep would be significantly lower with a slight increase in windage.
In the opposite direction, 5mm New England Ropes STS-HSR with its breaking strength of 9,300 pounds would be loaded at 16.3% of its breaking strength. While windage would be significantly less, the creep would be considerably higher than with the other two options.
Additional windage is from thicker stays is not the end of the world, though they rope itself is more costly. Choosing a size that offers you the resistance to creep and windage you are comfortable with depends on your ability and willingness to tune the rigging. If you choose a very small stay that will creep, you will need to tune it more often. If you choose a thicker stay, it will cost more and be more windage, but it will hardly creep at all.