One of the best locations for autumn colours in New Zealand would have to be around the Arrow River and Gorge next to Arrowtown. The miners and settlers planted deciduous trees throughout the area, which have reseeded and become well-established. Compared to the mostly evergreen native trees of Aotearoa, they are quite a striking sight.
As far as Sawpit Gully and Tobins Track, the Gorge Track is also an excellent short walk in its own right. It offers a bit of gold mining history, including stone building ruins and views high above the river. If you have an extra 2 to 3 hours, you can continue the loops around Sawpit or Tobins. Both options are highly recommended.
Arrowtown is also home to the autumn festival, normally around the 3rd week of April.