Elegant Editor-Only Script Execution in Unity3d

Lately I’ve been doing a lot of Unity programming. I love how easy it is to develop editor tools for my designers and how seamlessly these tools tend to integrate with the runtime environment. My favourite feature is the [ExecuteInEditMode] attribute because it lets me write all kinds of funky real-time level generation tools. (Like extruding 3D meshes from 2D primitives!) But what this attribute sorely lacks is the ability to have code execute ONLY in edit mode, where performance isn’t such a big deal. By default code placed in the Update() method will execute both when the scene changes in edit mode AND during every frame of play mode, making it unwieldy for expensive operations like realtime mesh generation (or even a large number of tiny operations, like setting a renderer’s sorting layer).

Step 1: Execute Only In Edit Mode

Various hacks around the internet purport to solve this problem. Some suggest using conditional compilation directives like #if UNITY_EDITOR. This is helpful for platform-specific builds, but unfortunately code placed within these directives will still execute in the editor’s play mode (which is, of course, where 98% of testing happens). Another approach involves checking a global variable like Application.isPlaying before doing anything editor-specific, but this is less than ideal should you have hundreds of active scripts in your scene (the performance cost of calling Update() on that many components, even if that function contains only one boolean comparison, can add up surprisingly fast). I’ve also found, although I haven’t verified it experimentally, that isPlaying becomes rather unreliable when it mixes with big scenes, custom inspectors and ExecuteInEditMode scripts.

I now use a solution that makes use of both conditional compile tags and a little-known global variable in the EditorApplication class:

[ExecuteInEditMode]
public class CurveGenerator : MonoBehaviour {

#if UNITY_EDITOR
	void Update () {
		if(UnityEditor.EditorApplication.isPlayingOrWillChangePlaymode) {
			this.enabled = false;
		} else {
			// editor code here!
		}
	}
	#endif
#endif

}

Here’s how this works. In edit mode, isPlayingOrWillChangePlaymode will always return false; thus, the editor-only code located within that else statement will execute every time Update() is called (generally every time the scene changes). In play mode isPlayingOrWillChangePlaymode will always return true, causing the MonoBehaviour to disable itself on the very first Update() and cease hogging performance beyond the first frame. (This is especially helpful when you’re trying to use the profiler to isolate script bottlenecks.) In compiled versions of the game the Update() method won’t even exist, saving us the trouble of having to disable it. The result of all this is that you can run horrifyingly-expensive setup code in edit mode that costs virtually no performance during play, which arguably is the entire point of an IDE like Unity.

Now, I know what some of you are thinking: ‘Why not do away with all this ExecuteInEditMode voodoo and simply write a custom inspector? It will never run in play mode and you can use SerializedProperties and GUILayout and all that magic.’ Well, here’s the thing: Custom inspectors act only on currently-selected Unity object(s). Suppose you had a bunch of prefab instances, some of which had overridden properties, and you wanted every single one of them to regenerate their content each time you modified the parent prefab? What if some of your generator scripts modify other generator scripts, and you needed those ones to be responsible for their own regeneration even when not a single one of them is selected (and therefore their custom inspector code is completely unloaded)? What if those scripts used prefabs? ExecuteInEditMode provides an understandable, reliable, and relatively efficient way to let scripts respond to any change in state from anywhere in the IDE. Inspectors will never do that for you.

Step 2: Only Regenerating When You REALLY Need To

At a certain point your generation algorithms will get so horribly expensive that it becomes impossible to simply run them over and over again with every single editor update. Say, for example, you’re generating a family of procedural meshes that wrap along a set of real-time-editable spline curves; say your level designer then decides to make 300 spline curves in a single scene. Once again, custom inspectors are good at detecting modifications to a currently-selected object’s properties but they are next to useless when it comes to non-inspector-based changes.

My first stab at this problem involved storing information about the last time my scripts ran each time they regenerated their contents, so that they could tell when this information changed and another regeneration was therefore necessary. Sadly this solution did not account for prefabs because the information got saved into the prefab itself. Information about an individual instance’s procedurally-generated stuff should never transfer from one instance to another, since often the things these instances generate need to reference information outside of the prefab (we may, for example, wish to apply the same line style to three separate spline curves); otherwise we would encounter all manner of craziness to do with random properties overriding one another and being set incorrectly from above.

What we need here is something ridiculously specific: A data structure that NEVER stores itself in prefabs but which can nonetheless be associated with individual prefab instances and referenced on each update to determine whether they should run their horribly expensive subroutines over again. We’d also like this structure to serialize and de-serialize properly so that our scenes can be saved and loaded with cleanliness and efficiency. Most of all, we’d like a flexible solution applicable to many different components that is relatively non-hacky and therefore unlikely to break after every Unity update. These requirements, quite frankly, are insane. Fortunately the Unity editor is no stranger to insanity.

Have you looked very deeply into these ScriptableObject things? They’re kind of bonkers. They can reside in scenes, but unless explicitly saved as .asset files they cannot reside in prefabs. They serialize nicely even when subclassed, which is something regular serializable objects don’t do. They work in the editor as well as during runtime. And if you are a generics geek like me you’ll be interested to note that though it is impossible to instantiate a generically-typed ScriptableObject directly, it is very possible to instantiate one of its concretely-typed subclasses. For an application programmer like me this makes them the stuff of magic. Check this thing out:

public class Regenerator<T> : ScriptableObject {

	public UnityEngine.Object Parent = null;

	public virtual bool UpdateResult(T info) {
		return false;
	}

	public static bool Refresh<C>(UnityEngine.Object parent, ref C lastResult, T newInfo) where C : Regenerator<T> {
		if(lastResult == null || lastResult.Parent != parent) {
			lastResult = ScriptableObject.CreateInstance();
			lastResult.Parent = parent;
		}

		return lastResult.UpdateResult(newInfo);
	}

}

This is essentially an abstract parent class whose UpdateResult method is responsible for comparing information about some script’s previous state with information about its current one, updating this information internally and then returning ‘true’ if the two don’t match (and regenerating is therefore required.) The information in question might take the form of a data structure like this one, which defines the diagonal length of a procedurally-generated quad mesh and provides an Equals method to help us determine whether one of these structures holds the same information as another:

[Serializable]
public class QuadGeneratorInfo
{
	public float Length = 1;

	public bool Equals(QuadGeneratorInfo info)
	{
		if(info == null)
			return false;

		return info.Length == Length;
	}

	public QuadGeneratorInfo Clone()
	{
		return (QuadGeneratorInfo)this.MemberwiseClone();
	}
}

By subclassing Regenerator we can then exploit our Equals and Clone methods to reduce a whole bunch of complicated-sounding update logic into about three lines of code:

public class QuadGeneratorResult : Regenerator<QuadGeneratorInfo>
{
	[SerializeField]
	private QuadGeneratorInfo Result;

	public override bool UpdateResult(QuadGeneratorInfo info)
	{
		if(!info.Equals(Result)) {
			Result = info.Clone();
			return true;
		}

		return false;
	}
}

The beauty of C# generics allows us to condense all this wizardry into one very expressive line of code. Remember that scary-looking static function three code blocks back? Here’s a sample MonoBehaviour combining that monstrosity alongside our previously-discussed editor-only update function AND our spiffy new QuadGeneratorResult class. Pay special attention to the first line of the Refresh method:

[ExecuteInEditMode, RequireComponent(typeof(MeshFilter)), RequireComponent(typeof(MeshRenderer))]
public class QuadGenerator : MonoBehaviour, RefreshTarget
{
	public QuadGeneratorInfo Info = new QuadGeneratorInfo();

	[HideInInspector]
	private Mesh QuadMesh;

	[SerializeField, HideInInspector]
	private QuadGeneratorResult LastResult = null;

	#if UNITY_EDITOR
	void Update () {
		if(UnityEditor.EditorApplication.isPlayingOrWillChangePlaymode) {
			this.enabled = false;
		} else {
			Refresh(false);
		}
	}
	#endif

	public bool Refresh(bool forceRefresh)
	{
		if(QuadGeneratorResult.Refresh(this, ref LastResult, Info) || forceRefresh) {
			// (horribly-expensive mesh-generating stuff here)
			return true;
		}

		return false;
	}
}

Where at first there were as many as two generic types, the magic of concrete subclassing reduces that number to zero. The resultant code, I dare say, makes perfect sense. Whenever the length property changes (and ONLY when this property changes) during edit mode, be it through inspectors, prefabs or some other editor script, our mesh will recreate itself accordingly. Even in play mode regeneration is still possible by calling Refresh; if you wanted you could even encapsulate Info in a property and call refresh any time another script modifies it.

This pattern is highly extensible; you could use it to mitigate any sort of expensive recalculation, in-editor or otherwise. Or if you wanted to you could just take the ‘Execute ONLY in Edit Mode’ stuff and do something else with that. I’ve fleshed this all out with some actual, functional Unity code that you can download here; it’s MIT licensed, meaning if you want to you can modify it or use it commercially or whatever. Let me know whether you find it useful!