This should be a toggle in the settings for a project.

Right now, this causes issues with more advanced projects, for example, ones with a file watcher (like a node server).