Hey, very cool!
Loved your screencast, makes it a million times easier to understand what you mean. I'm still halfway through it, but let me start replying already:
A major source of confusion (and understandably so) has to do with the several components of the Winstep Xtreme suite, so let me see if I clear that up.
You can divide Xtreme into the following:
1. Shelf (tabbed dock).
2. Nexus Docks.
3. Destkop Modules.
4. Task Management (taskbar), menus and hotspots.
5. Start Menu Organizer.
In order to save resources (and for historical reasons which I'll explain below), the Shelf, Nexus docks and Desktop Modules (so components 1 to 3) have been merged into a single executable (WorkShelf.exe) and share a common Preferences GUI under the WorkShelf logo. On the other hand, since you can have more than one dock, each dock also has its own individual Properties panel. Same for the Shelf (although currently you can only have one Shelf, as you've probably guessed by now, the plan is to be able to have as many as you want in the future, just like docks).
So, under Winstep Xtreme, WorkShelf = Nexus + Shelf + Desktop Modules.
NextSTART (NextSTART.exe), on the other hand, is the application responsible for the alternate skinnable taskbar, customizable menus and hotspots (component 4 above).
The Start Menu Organizer (WsOrganizer.exe), is also a stand alone application dedicated to organizing the Windows Start Menu.
One of the reasons this ended up like this also has to do with the history of Winstep. Winstep started back in 1999 with NextSTART, which began as a simple menu system to emulate the look of other OSs and complement/replace the Windows Start Menu. Eventually NextSTART evolved into a complex and powerful application capable of not only replacing the Windows Start Menu but also the Windows taskbar.
Some time later, WorkShelf was also released - the Shelf. The idea to add free-form Desktop Modules/Widgets to WorkShelf to complement the in-shelf icon modules came a short time later too. To keep things simple and save resources, because Desktop and In-Shelf modules share a lot of the code, it was decided to have everything running from the same executable instead of splitting each module into individual executables.
A much antecipated (at the time) dock type application which would be called 'Nexus' was also planned from the very begining, but it remained vaporware for many years.
At the time, NextSTART and WorkShelf were sold as individual applications. It wasn't until some time later that the Nexus dock made it's appearance and the whole thing started being sold together as 'Winstep Xtreme'. Again, to keep things simple and save resources, the Nexus dock, which at first was supposed to be a separate application, became it too part of WorkShelf (and by extension also a component of Winstep Xtreme).
Things stayed like this until early 2009, when a decision was made to market a stand-alone, free version, of the Nexus dock that is part of Xtreme.
This free version of Nexus was so successful that it created the need for a new paid product: Nexus Ultimate. Nexus Ultimate is essentially WorkShelf minus the Desktop modules, and it is an important product because Winstep Xtreme is just 'too much' for some users.
So now you know how convoluted these things can get. As they say, hindsight always gives you 20/20 vision, unfortunately it's only after the fact.
When you run Winstep Xtreme, besides getting the dock, desktop modules and the Shelf, you also get a new skinnable taskbar that replaces the normal Windows taskbar. As I explained above, this replacement taskbar is provided by NextSTART.
Now, when you say you are closing Nexus, if you look closely what you are actually doing is exiting Winstep Xtreme. This closes down WorkShelf *and* NextSTART.
If you really don't want a taskbar at all, then click on the Start Button of the NextSTART taskbar, open the Settings sub-menu and click on 'Preferences'. This will open NextSTART Preferences. Navigate to the Taskbar tab in Preferences and uncheck 'Allow NextSTART to replace the Windows taskbar'. This will bring your Windows taskbar back, which can then be hidden via the appropriate setting in the General tab.
Of course, if what you really want to do is auto-hide the NextSTART taskbar, instead of clearing the 'Allow NextSTART to replace...' setting, click on the Taskbar Settings button, enable 'Auto-Hide after a short delay', click the Advanced button, and select 'Hide into screen edge...'.
As you are probably begining to realize, Winstep Xtreme is very easy to use if you stick to the defaults, but once you start tinkering with it, behind that simplicity is an extremely powerful, flexible, and complex suite of applications. Winstep Xtreme is a power user wet dream because everything is customizable - the drawback is that once you go beyond the basics, you run into a learning curve.
For instance, context menus in WorkShelf (Nexus+Shelf+Desktop modules) are skinned by NextSTART. When you change the NextSTART theme, you also change the skin of the context menus in WorkShelf (probably the cause of the problem you run into). You can change this behavior by opening WorkShelf Preferences -> Themes -> Advanced -> Context Menu Options.
As for having the very top of the Shelf tabs still visible when you collapse the Shelf, that is by design. The Windows taskbar does exactly the same thing when it auto-hides, so you know it's there.
The other issue is with the way the NextSTART taskbar hides by default. Instead of hiding down into the screen edge, it slides to the side until only the handle is visible. Again, you can change this behavior by following the instructions I gave you above to auto-hide the taskbar. Unfortunately you will still have an empty space (this and the taskbar sliding instead of hiding into the screen edge by default have to do with a problem with the Windows taskbar still reserving screen space even when hidden, which has been solved for the next release). Since your *Windows* taskbar is already set to auto-hide, here is what you can do to solve the problem: in the Taskbar Settings dialog, clear the 'prevent maximized windows from overlapping the taskbar' and enable 'Always keep above other windows'.
The next release will handle these issues better.
As for the tab mouse over: you need to close the Shelf Properties dialog first for it to work - and keep in mind there is a short delay (as in half a second) before the tab is automatically selected when you mouse over it - keep the cursor still.
Lack of Taskbar tab: I think you already figured out why by now. You're looking in the wrong Preferences screen. Besides the Settings sub-menu of the main NextSTART menu, you can also open NextSTART Preferences through the system tray icon or by pressing CTRL+F12.
Magnification not working issue after applying the Unlighted theme: looks like a bug. Errr, let me rephrase that: It is a bug. The dock thinks it's hidden when it's obviously not (which is why it wasn't auto-hidding either) and then when you bumped the top screen edge it popped out again, which made the magnification work as normal.
The theme change that confused you: instead of changing the theme just for the Shelf (i.e.; via the Shelf Properties dialog) you changed the theme via the WorkShelf themes tab with 'WorkShelf (All Items)' selected in the combo box. This changes the theme for EVERYTHING (docks, Shelf, desktop modules) including NextSTART's menus and taskbar, because the whole suite is integrated (i.e.; change the global theme in WorkShelf, NextSTART will also assume the new look - and vice-versa).
For your information, you can change this behavior in WorkShelf Preferences -> Themes tab -> Advanced -> Winstep Theme Integration. Also, not only can you change the individual theme of each object (docks, shelf, desktop modules, etc...) but it's also possible to lock the current theme of an object so it remains the same even when you change the global theme.
Now, keep in mind that Winstep Xtreme can change not only the way the Windows User Interface *looks* but also the way it *feels* (works): you changed the theme to the NeXT OS theme - if you're familiar with that Operating System, which was created by Steve Jobs when he left Apple and before he went back to it, it doesn't have a taskbar or a Start Button. It has task buttons (which was the stuff at the top of your screen which you were wondering what it was).
The easiest way to sort the 'mess' you got into was to change the current 'WorkShelf (All Items)' theme to Aero Low Res (which would have restored the previous look of your context menus and the NextSTART taskbar), and then change the individual themes of the Shelf and the dock.
The reason why the Reset Settings button in Troubleshooting did not work completely was because that resets the settings for the WorkShelf application only. To reset the settings for NextSTART, you would have to done the same in NextSTART Preferences (that missing Preferences screen explains a lot, heh? I bet a lot of things are starting to make sense now).
There is a LOT of stuff hidden behind the surface of Winstep Xtreme which you still don't know about - but keep tinkering with it and it will soon start to make a lot of sense. By then you will have realized how powerful this stuff really is.
Of course, you just started using Winstep Xtreme and you are already diving head deep, unlike most users. Well, at least that is part of the fun.