Wow! Bunch of replies here.
If you have a cylindrical or tubular section in the middle without a keyway then the math is pretty simple. I have a textbook I can pull out and try to work through some of the math with you.
The core principle for all materials is the stress-strain curve (see image below). Stress is the force per unit area in the material and strain is the deflection per unit length.
As the applied force (torque in your case) increases, the stress in the material rises. For simple shapes, the calculation of stress is easy, but adding a keyway to a shaft significantly increases the stress concentration at the corners of the keyway. Anyways, as stress goes up the material immediately deforms and starts to stretch or twist. As long as the stress stays below the yield strength the material will return to the original shape when the applied load is removed (just like a spring). If the applied stress goes above the yield strength of the material, a permanent deformation will occur. The material must still spring back the equivalent amount of strain before the yield strength; you notice this when working on sheet metal and you must deal with spring back.
Anyways as more and more load is applied, the stress in the material strain hardens until it reaches the Ultimate Strength. At this point, if the applied load remains, the material rapidly fails until fracture occurs.
In brittle materials (glass, ceramics, etc.) the fracture strength is lower than the yield strength and they will crack before plastically deforming.
Here is a great link that shows the formulas for calculating stress in common shaft shapes.
The torsion of solid or hollow shafts - Polar Moment of Inertia of Area.
www.engineeringtoolbox.com
If you stay below the Yield strength, the shaft will deform elastically (not permanent), and the deformation will be linear to the applied force, making a great torque meter using two encoders as you showed above. The relationship for stress/strain below the yield strength is Youngs modulus, a published value for most materials.
One other value you need to be aware of is the fatigue strength. This is a value of stress (typically below the yield strength) where repeated cycling of the load will tend to generate and grow cracks. Cracks have a very sharp tip which will concentrate the applied stress at a stress concentration (or stress riser). This causes local deformation and crack propagation. You typically want to design below this limit in designs that have cycling loads applied to them.
en.wikipedia.org
What I find very interesting is that steel, titanium, and some other metals have a defined endurance limit where they will weaken from repeated load cycles, but only to a point, from there they retain their remaining strength for infinite cycles of the same applied stress.
Other metals like copper and aluminum have no endurance limit and will eventually fail under even the smallest cycling loads (after a crazy number of cycles).