It seems to me that a lot of game developers jump head long into setting up their game engines to be based around writing gameplay code in some scripting language without any sensible reason to. It just seems like the thing to do, right? Everyone does it so obviously we should to! Often times these languages offer no real productivity improvements over native C++ but still require significant work creating such systems, binding them with the engines and maintaining the interface with all gameplay systems. This also comes at the cost of being able to use a top notch debugger (like Visual Studio) in place of either a non-existant or inferior one and at the cost of a massive performance hit for any code executed inside script (anywhere between 20x slower to 200x slower depending on the VM).
So, what are the benefit FOR scripting (feel free to ping me with anything I missed):
1) You can sandbox in your gameplay programmers so they don't crash your game. This is nice in theory, but in practice they still manage to crash and/or hang the game just fine. Any language that is flexible enough will also not be fool proof enough. There just simply isn't time to make sure all the entry points back to native code can't do anything stupid.
2) It's easier to program for because its a script. This argument never made sense to me. On top of there being an inferior debugging environment, any game studio is not going to hire a programmer/script that doesn't have significant C++ experience. So on top of throwing away that experience they are getting tossed onto most likely a new language for them to learn. Wouldn't it be more efficient just to use the language EVERYONE is already using and as a bonus have it run at native speeds? Also, I have seen a few different game scripting languages that are basically just VM versions of C/C++... which I guess is an attempt at making it more familiar to C++ programmers but really just completely invalidates the argument that it can somehow be easier, since it basically is the same at this point.
3) Iteration time is quicker with scripts. This is unfortunately true sometimes, but only because IMO the project is not being properly maintained and thus build times are completely absurd. I have seen large projects take 40 minutes to compile and 10 minutes to link... but I have also seen LARGER projects take just a few seconds to incrementally compile+link. To me it seems easier to fix the build rather than setting up the bindings between native and script. Another option is sectioning off gameplay code into a DLL, which is not ideal, but more ideal than a VM.
And the arguments against:
1) Slow as shart. Even the best VMs can still be 20x slower... think about that, 20 times slower. Where else in game development has such a massive perf hit been acceptable?
2) Memory, this varies a lot, but some off the shelf scripting languages (like python) can be quite costly on this front.
3) VC++ has a better debugger and all your programmers already know C++. There is also loads and loads of 3rd party libraries/middleware available to use directly in C++ with no binding craziness required.
4) Your engine team saves loads of time not having to maintain bindings to the scripting system.
5) You have LESS code without a scripting system. So your build times are less, you have less code to maintain and potentially less bugs.
So, How does one make such a transition from VM to Native without causing complete insanity?
1) Don't do this mid project, thats a terrible idea, what are you thinking?! This is something to consider for next project.
2) Instead of making bindings for everything... make clean/safe APIs. This will both improve internal engine productivity/safety but make it easier and safer for game code to access such systems.
3) Have an incredibly anal memory allocator that slaps you when you leak memory, go above budget, or just grow too quickly. Make sure you have tools (either internally developer like Excel dumps, or externally developed like RAD's Dresscode or Deja's Insight) to keep tabs on who the big spenders are. Odds are you already have all this infrastructure, and odds are the game scripters can already abuse stuff in script. But if you have a lot of junior coders its best to keep track of them early and teach them proper usage rather than trying to clean stuff up later.
4) Have strong discipline across your engineering org... even scripters. Stop letting the gameplay guys write terrible code just because its script and you don't need to look at it. Trust me, it was an unmaintainable mess before it was C++ too, you just were too busy ignoring that problem. Make sure they follow the coding standard, they don't hack random parts of the code base and that they follow some sort of generalized actor or component paradigm rather than just making shit up on the fly. Remember, they were somewhat constrained before in what they could do in a scripting language, make sure they don't think the transition to C++ is license to do whatever the hell they want. Migrate from technical constraints to manual constraints enforced by the leads. It's not that hard, and they will learn quickly.
So, basically this rant was about game studio leads pushing off the responsibility of mentoring and managing junior programmers and instead just tossing them in a sandbox. Sounds harsh but that is one way of looking at it I suppose. My argument is that with strong discipline and good practices on the API front lots of time and energy can be saved by ditching a complex scripting language and in the process gain untold amounts of performance back.
Not to confuse you all, but there is still probably one potentially good place for "scripting"... and that is per-level sequences. I would prefer to not think of these as scripts/code but rather as data. In essence this is stuff authored by level designers or cutscene animators, not by your coding staff. In Unreal Engine this is normally the territory of Kismet and Matinee and I think visual sequencing languages is the real king here. If budget or schedule does not allow for the development of such a visual system I think a simple batch file like scripting system is all that is needed. Maybe there is already middleware out for this though?