Arrows are tough. Tougher than bows in some ways. I've used two techniques, the standard one I was taught and something I made up.
1) build a little tool with a hole the size of the arrow shaft, steam the arrow for a bit and slide the tool over bending the spots needed to straighten it. You'll have to go over it several times, steaming in between, because after a couple minutes it cools down too much to keep the corrections you make. I work from one end to the other, trying to get the part I'm working on straight and then keep that part out of the steam on subsequent passes because if it's re-steamed it will return to something closer to it's original state.
2) I just made this second method up, but I got sick of straightening arrows the normal way and gave this a shot and it seems to work pretty well. Get a couple of pieces of cordage and a shaft a bit longer than the arrow you want to make. Find two trees or anything solid about one and a half times the length of your shaft. On the one end you are going to tie the cordage off to the end of the shaft and to a tree. On the other end, you are going to tie the piece of cordage off in the middle of the length and run both ends to your other tree. If you twist a piece of wood in the center of this length of cordage you'll make what is known as a Spanish windlass. Basically, the twisting shortens the length of cordage so that you have tension on the arrow shaft from both ends. Steam the arrow and stretch it. The shortest distance between two points is a straight line, so as you increase the tension the steamed arrow shaft straightens. Keep the tension on until after the shaft cools and it will stay straight.
You'll get bends coming back if the temperature or humidity changes, so try to store your arrows outside (out of the rain, of course, but don't straighten in a dry house and then expect them to stay straight outside in the rain.)