Thoughtful science communication is essential for the success of collaborative, transdisciplinary environmental research. We present an innovative evaluation of a four-year pilot project that took a highly engaged and collaborative approach to managing excess nutrients in the Cape Cod region of Massachusetts, USA. The evaluation approach included mid- and end-of-project interviews with researchers and project partners and a reflection from the lead science communication researcher. We found that an effective science communication evaluation needs to be (1) adaptive, (2) multistage, (3) holistic and objective-based, and (4) democratic and reflexive. Results demonstrate that formative and end-of-project science communication evaluation of research projects lead to improved engagement that better meets all collaborators' needs.