A pedestrian-only suspension bridge crosses the lake, and provides a good photo opportunity for the lake. There are some hotels and apartments across the lake, and a few older food stalls, but otherwise there's little of interest there for day-trippers from Taipei.
Pedal boats are available for hire, and are especially popular among young people, especially on dates. It's also possible to hike up into the mountains, and a quaint little pedestrian boat operates further downstream to carry locals or hikers across the river.
There are no lockers anywhere near Bitan. So leave anything heavy in your hotel or in lockers at Taipei Main Station. You may also want to consider what to do with any electronic devices or other valuables if you intend to take a boat out onto the lake.