Note that the first entry in the “Artist” column is – (All Artists…)
However the “Album” column lacks the – (All Albums…) entry.
That has never been possible. The (All nnn albums…) only shows if you have a specific artist selected. Otherwise I would have to list all songs from all albums from all artists which is a big non-useful list.