In my opinion and again this is my opinion I am no expert on this you are asking a 8 bit micro to do the work of a 16 or 32 bit micro.
My version of Arduino indexer for dummies does what I ( My daughter) needs it to do for the laser engraver. Rotate item's 90, 180 , 270 deg from start and not to cut gears.
I am sorry I am no help but there are others on here with more knowledge than me that can.
Keep at it I am sure you will get it done.
( Looks like you need to tension your timing belt too. ) The timing belt is fine but you are right in the photo it looks like its going to fall off.
stepper resolution = 200 full steps per revolution (
1.8° step angle) / half step controller setting = 400 steps per revolution x 2 for 2 to 1 timing pulleys = 800 steps per revolution. So for 180 deg as shown in pic #1 400 out of 800 = 180 deg.
Here is food for thought I wounder in your application a 3 phase stepper motor would work better then the standard 2 phase stepper ?