optimus you got quite a lot of replies but I will add mine anyway: it depends on what you want to do, but IMHO if you are future oriented UDMF is way to go, others might be a loss of time.
Despite everything already said, I will add my two cents: you have another three elements to consider here: commercial timeline, opensouce timeline, and the future.
Original Doom is product of commercial timeline - because time restraints, original reasearch and whatnot the internal game formats are a) not optimal, b) there existed nothing similar to doom prior doom to use as example as solution for problem of data formats.
Actually I personally believe that the concept of engine you have these days (unreal, unity, gamemaker and whatnot) is direct result of doom existing. Of course all prior games used some way to organize data, but from developer standpoint many of that data are simple tables baked directly into the game executable.
Doom is the first modern game engine, a first real external data driven machine of it's kind (besides few exceptions). The concept of IWAD and PWAD is well thought out an ingeniously "simple" allowing incredible flexibility. So that part went good. Thank to this you can now have standalone engines and games being "just" their datapacks (Unreal/Unity case).
In essence for all this data driven design to work the doom (and any other data driven game engine) is also some kind of "mini os" of it's own. It requires more general IO routines than just some dumb tables (doom still uses internal tables for monsters and whatnot).
This is where doom is BAD (in the hindsight of course). The internal format of WADs is horrific: it is kinda a filesystem, but the entities inside are called lumps instead of files, there are no real directories to delimit these internal "files"/lumps (only some hacks with empty "files"/lumps like MAP , PSTART, PEND and so on), all the lumps are stored in the main directory and can only have very short names etc. Same shit happened to internal map structures (your first clue).
Quake 1 did much better with the .pak format that is used instead and it has real files now, with case sensitive names, true directories etc. But the format is custom, so you still need custom tools to pack and unpack it and whatnot. Also there are some years between Doom and Quake, so you see how much id learned in the meantime.
So finally, in Quake 3, Carmack "solved" this problem once for all, the game data file is just zip with renamed extension and that's it. All you need is zip packer and you can pack a Quake 3 release yourself.
Why am I talking about it? All modern advanced sourceports support zip aka pk3 and some even 7zip aka pk7. Both package formats have pretty good compression ratio vs usability properties and shitload of tools to optimize.
With regard to tools. Most modders are not that smart these days, as it does not matter at current performance characteristcs anymore, but if I remember correctly, both zip and 7zip support having 1:1 unpacked files in the package, while other files are packed. With more advanced zipping tool with scripted execution you could easily make a zip pack where internal oggs, mp3s and pngs are unpacked.
These files are already densely packed on their own, so packing them again with zip is useless and only adds cpu bloat to load (which nobody notices anymore with muticores and SSDs). PNGs are zip packed themseleves and jpgs, mp3s, oggs and videos are very information dense and thus unpackable, so avoiding packing all that might mean faster loading, shaving couple of msecs from load times and so on. Would be interesting test to make comparisons if such packing micromanagement would be worth the effort.
On Zdoom wiki there are translation rules for all the usual IWAD entities and their packing paths in the pk3 scheme.
So just by switching to packed format like pk3, you gain much more than new format - more flexibility, optimization with potential tools and whatnot. Only thing where it hurts you, is when you target some less capable ports, which are less capable in the name of compatibility, trueness to original doom, or some other philoshophic ground.
pk3 was "hack" intended for Quake 3, but thanks to opensource doom timeline it somehow found it's way into the doom ports and pk3 packed relaases are much more futureproof than pure wad ones thanks to it.
Now back to UDMF issue: HEXEN and BOOM added insane amount of cool stuff to doom engine, ACS and HUBS of HEXEN are most favorite of mine. The "problem" here is these things were done in commercial timeline.
Commercial timeline is very different from OSS (open soruce one): there "realtime"
hard deadlines and so on. So to integrate ACS with original doom map format both had to be extended in semi hacky ways. And that is exactly what happened with HEXEN.
This means while it all works - it's bit flaky and fighty. But it worked well enough for commercial releases. Enter open source timeline now.
Finally a ZDOOM came and modders wanted to have all these cool HEXEN features in classic doom maps. First modifications of zdoom happened in commercial timelineish matter (despite bein open, we as a commmunity were still learning, not many can dive into the engine code) and that meant some baskerville bastard child of HEXEN and DOOM, kind of hexendoom hack, which kinda works but is in essence a mess too.
And then with GL and whatnot many new and newer features made their way into the engines. Now doom engines of sourceports are so capable that all kinds of crazy stuff is possible: 3d floors and dynamic lightning and what have you - but now these old hacky map formats are the limiting factor.
Now, designing good extendable binary format is quite hard endeavour, just ask matroska guys, and with each passing generation, programmers are getting stupider. Not only that, but in hindsight it turns out, that pure text is very compressible format especially when compressed by the like of zips an 7zips of the world. Many times 7zip compressed text format beats the shit out of 7zip compressed binary format storing the same information. It has something to do with how compressor can pack letters instead of binary chunks.
Of course not everything is exceptionally well compressed when expressed in text form - ie images sounds and similar.
Anyway at that time zdoom went with very radical solution. Just like Doom 3 did when carmack ditched binary formats in favor of text based ones for model defintions and animation, which sounded insane but turned out well, Zdoom decided to express map data as pure text, and that is all what UDMF is.
Now why does it matter? Due to nature of textual data it is easy to add new stuff to it.
You add something new to a wall in your engine? Some new property?
Well, in binary format, the format is either so well thought (like matroska's EBML) that you can do it similar way to text format (which in doom it isn't as we discussed), or you must redesign your whole binary format, so that it is able to capture that new feature.
You need to change binary structure of a wall in a map to new binary structure of a wall, such where the field for the new feature of a wall is added to the wall structure (as these things act as a kind of slot that must be added into data). That format redesign will break your set of all maps into two sets: before the redesign and after. And each time you add something new, the additions break the map format and the cycle repeats. Now add several features at one time and as you see, as features are added willy nilly, this has potential to fragment the whole mapping space into many shard of maps in islands of formats supported only by certain sourceports. Ie disaster of babylonian proportions.
In text format such change is just a new line, or set of lines, with new properties understood and added to the text and that's it. That's why UDMF is kind of big deal TM. Now you can have a millions of ports with millions of features and hopefully each will at least load something. Map might be broken, but unless the features added are really crazy, something should load. The format becomes extensible.
So that's what some posters explained to you: in older map formats there is a chance you want to apply several features to an object that cannot be applied on it at the physical map format level. Engine is easily capable of applying all those features to a wall directly for example, but map format is unable to express them, so to have them in a game you must either revert to some battery of hacks or the thing is impossible to implement even.
Now with UDMF this is easily possible, because the map format can "swallow the load" imposed on it. UDMF is in fact so potent, it can actually express almost all the features of all the doom engines in a single map, that why it is called "Universal". UDMF thus is kind of futureproof and more efficient for technologically heavy maps than any other doom format.
Can you achieve the same with older formats? Many times yes, but if you are often hitting the limits or you often see can do same stuff in UDMF more efficiently - your project is probably more future oriented and want to use that format more often.
Things like UDMF can often be conceived only in hindsight and often only in open source timeline, where there is enough time to ponder about deep philosophical problems, that are impossible to tackle at commercial timelines.
So now what to takeaway from all this.
Due to UDMF own nature it is "the future" (extensibility, future proofness), and as such it requires never doom tech. So if you are mapping for older less capable engines it is not a format for you. If you are hitting the limits often and are okay with limiting your release for UDMF capable ports only, it might be a very good choice.
hard deadlines and so on. So to integrate ACS with original doom map format both had to be extended in semi hacky ways. And that is exactly what happened with HEXEN.
