@Missionary I can reproduce this issue with clicking the color twice, now! It only happens in Objects of Type “Note”. I tested it before with a Page. The issue is no longer present if you change the Object Type to something else and then back to a Note.
Do you want to file a bug report, or shall I create one for you?
For the TOC in the sidebar, this seems to be a relevant and nice feature request: