Hi Zorro, you have put forward some interesting ideas in this thread. I'd like to contribute some of my opinions..
THE CODE
---------
As others have pointed out, remaining compatible (we have precision DOOM demo playback compatibility, for example) would be almost impossible without paying homage to the quirks of the original code. This means any roots-up reworking is going to be difficult. Yes, we can build a better game engine easily, but it won't be DOOM. For example, merely changing the order of tickers would break demo playback; taking the engine apart and putting it together "brick by brick" will do so to an unmanageable extent.
Given these restraints, I disagree with you when you say that the engine is a "mess" or a "jenga tower". For a start, Odamex code has amazing stability and portability compared to other ports. The code maintains all its traditional characteristics while accommodating newer techniques where appropriate (e.g. STL for string handling). The improvements have been happening over a long time (we have gone from 150,000 lines to 95,000 lines while improving features). Have a play with the old csDoom versions to understand why Odamex has moved miles ahead.
That's not to say that there aren't problems. Close module coupling and cohesion make modification more difficult than it should be. There is a lot of work to be done, adding proper spectator support and demo playback are two of the big modification challenges.
DEVELOPMENT
-------------
The major reason for slow development is a lack of developer input. That includes me, as I happen to be travelling the country in search of a job while working on a 100 page thesis. Other members of the development team also have important real-world business to take care of. Hopefully we will all have more time over the summer holidays.
We often have people proposing various changes and restructurings. There are plenty of people pulling us in different directions. Plenty of people starting their own ports (and rarely, if ever, finishing them). If you think you can significantly improve the code by rewriting a large part of it, I am quite happy for you to branch the project and look forward to see what you come up with! We'll even host the branch on our SVN.
However, what we really need is dedicated coder time. Not someone to pull in yet another tactical direction, but along with our common strategic goals. A developer that trusts our management and strengthens our team. Someone that actually does the work. Someone who can dig their teeth into a specific tough problem and not let go until it is solved (which may take months of hard work). That's how things get done, how to get credit. If anyone wants such a challenge, just ask :)
We don't need developers who can talk the talk. We don't need developers who can walk the walk. We need developers who walk the walk.
EXTENSIBILITY
------------
I have been thinking about the 'extensibility' problem and how it can be dealt with. The sad truth is that everyone wants some specific mod/tweak and they are not inherently compatible. If we follow all the suggestions, the engine will sprawl with millions of special CVARS and console commands.
The best answer I can come up with is a move towards pure scripting. A translation of the core game logic into Python/Tcl/Lua/<insert language>. This will offload development to whoever wants the change, rather than the Odamex devs. We can then have a classic engine, a CTF engine, and etc. Load them up like a lump and run them. Someone wants flag dropping, silent bfg and no strafe running? Just modify the engine and load the lump onto your server. Client will download the lump and run it. Odamex becomes the technical component that does graphics, sound, engine interpretation, and network connectivity while the pure game code is moved to the WADs. Truly a new standard.
Another benefit of this is, potentially, cheat prevention. A scripted engine can be quickly and subtly modified to test whether it is being circumvented. There are associated security problems of letting downloaded code run, but hey.. your browser runs javascript already. Lots of other questions.. What language should be used? Where do we draw the line between core and script?
Big project to do, but I think it is the best way forward.
NETCODE
--------
A scripted approach will also reduce the need for anti-cheat network code, which many people have been complaining about.
The client and server both use a working engine. This allows for both SP and MP. The client has a copy of the engine for prediction capability. To start with, we had two separate copies of the engine, one in client and one in the server. This has now been moved into a single 'common' engine. That took serious effort. Some lines have been moving around as a result. It may still look messy, but it is an improvement.
The chief reason for a 'blurry' line between client and server is perceived performance. In reality, neither client nor the server replies that the client gets are accurate, in terms of timing. The prior is ping/2 ahead, the latter is ping/2 behind. The player's weapon sound is played at the prior, the bullet hit is actually shown by the latter. Theoretically, both should happen at the same time, the client should delay the sound and do the bullet hit before a server confirmation (which is checked after it arrives). Changes in this area are tricky as there is no common structure defined to deal with the events.
The major hurdle in netcode is verifiability. We need tools to visualise network data streams and how they influence game state. We need a netwide demo recording and playback tool that also records the network data, so that we can play back an entire network game (prom the point of view of multiple clients at the same time), and see where the game state is desynced. Otherwise it is like testing/fixing an electronic circuit without an oscilloscope/voltmeter. Only way to really show how ghost monsters and other problems originate.
There's also an annoying issue in current anti-wallhack code, primarily because the BSP sight checking code is slow and erroneous (p_sight.cpp). There is a much faster way to do it, but the task is tricky and we don't have a developer to take care of it (see DEVELOPMENT above).
MISC
----
Many problems exist outside the main engine domain (e.g. osx midi crashes due to buggy SDL_mixer, tricky wxWidgets versioning), so work should be done to prop up our dependencies in that case. There are some issues that just need people to fix them, especially in audio.
It does not need patchwork, it needs a reworking. It needs to be torn apart and redesigned as a client-server engine which lends itself to code reusability and sanity.
The only mistake is hotheaded development. If redesign was the answer, one of the other numerous ports would have already succeeded in our place. We should stop concentrating on development and start concentrating on testing. Regression testing and test driven development are the only way to stop making mistakes. We need testing tools that will, for any line change, tell us whether something (a demo, a visual, an audio, a protocol) breaks as a result.
Cheers,
Denis
