Subject category renaming

I think the first challenge I'll take on is adding prefix "Subject:" to the subject categories. It's occurred to me that all these category names appear verbatim in the DPLs on the subject pages (see for example Subject:History), and I'm not sure how I feel about all those book titles having "Book:" prefixed to them. Perhaps I just need to get used to the idea. I've been aware for years that there are lots of situations where one really wants to snarf a DPL and then somehow further process the results, which the DPL extension doesn't support at all. I have in mind to semi-automate it using my dialog tools once they're deployed but for now that's not yet an option. --Pi zero (discuss • contribs) 14:40, 4 September 2016 (UTC)

An advantage on your new system would be to automatically display {{Book search}} into the books categories only. JackPotte (discuss • contribs) 07:05, 22 September 2016 (UTC)

Good idea. --Pi zero (discuss • contribs) 09:04, 22 September 2016 (UTC)

Update: I've just been planning out the subject-categories campaign. As best I can figure, besides the categories themselves there are five key pages involved: Subject:Books by subject, {{Root subject}}, {{Subject page}}, {{Allbooks category}}, and {{Subjects}}; each of the five is transcluded on about two or three thousand pages (mostly, subject/category pages and the main pages of books). My basic plan is,  (1) modify the templates to support/populate the new as well as the old cats, and create the new ones;  (2) once the new cats are all available, modify things to use the new ones; and  (3) dispose of the old cats, not forgetting to do history merges when appropriate. --Pi zero (discuss • contribs) 16:14, 27 October 2016 (UTC)

Steps (1) and (2) appear to be complete. (3) is going to take a lot of time and meticulous effort, because it will require manually addressing the subject-categorization of every book, and book category, on the project (all 6390 — or so — of them). I can't really begrudge the labor, though, because most of it actually goes fairly quickly: the reason it's time-consuming is that it flushes into the open lots of pre-existing categorization errors, which then have to be fixed. So the whole project is going to be much better categorized by completion of this phase. --Pi zero (discuss • contribs) 04:23, 1 November 2016 (UTC)

I've disposed of 1/4 of the old subject categories. I'm rather encouraged about the infrastructure aspect of this; the most awkward/peculiar cases in the project are necessarily turning up and taking time-and-effort to fix, but it's all sorting-out that would need to be done for the next, and much larger, operation of renaming to book categories. Demonstrating that the larger plan is fundamentally coherent, as one phase aids the next. It's also so much easier to work with a mixed set of categories once you can tell at a glance that certain ones of them are subject categories, I'm quite encouraged about the wisdom of the larger renaming scheme. --Pi zero (discuss • contribs) 20:17, 12 November 2016 (UTC)

1/2 now. --Pi zero (discuss • contribs) 23:31, 23 November 2016 (UTC)

60% --Pi zero (discuss • contribs) 21:59, 2 December 2016 (UTC)

70% --Pi zero (discuss • contribs) 03:38, 6 December 2016 (UTC)

80% --Pi zero (discuss • contribs) 23:16, 9 December 2016 (UTC)

90% --Pi zero (discuss • contribs) 16:34, 16 December 2016 (UTC)

Done renaming the subject categories.

My sense, after that effort, is that the even larger effort to rename the book categories is just too large to undertake until I can bring semi-automation to bear on it; so I expect to put any near-future Wikibooks time into dialog/assistants. --Pi zero (discuss • contribs) 23:37, 19 December 2016 (UTC)

@Pi zero: This can probably be done with a bot that has admin rights. —Justin (koavf)❤T☮C☺M☯ 04:17, 20 December 2016 (UTC)

I wouldn't care to try it with a bot. Every case is likely to be different, requiring careful human attention. I hope I can use it as a test case for "growing" a semi-automated assistant, though. That wouldn't happen immediately, but I think separating the subject categories, which is now done, was the more urgent task. I'd rather put overhead effort into the semi-automated route rather than the bot route, because the effort into the semi-automated route also benefits future assistants. --Pi zero (discuss • contribs) 04:30, 20 December 2016 (UTC)

As a simple step to move things forward, I'm considering setting things up so that each subject category page recreates the content of the corresponding subject page. (I meant to set up {{subject category}} so that it would support this if the subject page wanted to cooperate; in theory, it should be just a matter of modifying {{subject page}} and {{root subject}} to provide the necessary content.) --Pi zero (discuss • contribs) 11:25, 7 March 2017 (UTC)

I've done this. All the subject categories should now mirror the full layout on the corresponding subject page; if any of them don't, ping me. --Pi zero (discuss • contribs) 05:35, 18 April 2017 (UTC)

Lessons learned from this operation: A useful test case for the sort of thing I'd like a meta-assistant to be able to help with. A large operation (though just within range for a single person to do manually), mostly repetitive but with occasional customizations required that seem to require a human being to recognize them. I imagine an admin conjuring up a meta-assistant, which then guides them through the process of setting up an assistant for the operation, at each step helping them to recognize and usefully provide information that the human operator should have in order to realize when a particular case wants customized treatment; and flexibility to handle such a case in a partly or entirely manual fashion, informing the manually intervening operator of what the uncustomized behavior would be and making it easy to get back into the semi-automated part of the operation once that manual intervention has been completed.

(Why am I so concerned with aspiring to meta-level support? Because I think we have gotten ourselves into a bad place by focusing so much on accomplishing specific tasks that we neglect the vital need to empower ordinary volunteers — the sort who should not be asked to deal with anything technical beyond wiki markup — to take charge of choosing what specific tasks should be done and providing the know-how for how to do them. Imho this is the sort of misfocus that has driven the Foundation's major software initiatives, leading to an ongoing shifting of infrastructure out of reach of ordinary volunteers. --Pi zero (discuss • contribs) 15:15, 18 April 2017 (UTC)

Book category renaming

I've somewhat revised my thinking about the size of the task of migrating the book-category names to use prefix Book:. Although it needs, I think, to be done slowly and carefully, it is probably not entirely beyond manual size. For any given book category, most of it can be achieved — but, often creating a mess that needs immediate attention by a human being — simply by renaming the book category (and its subsidiary categories) without leaving redirects, because {{BookCat}} and {{BOOKCATEGORY}} will figure out the move and recategorize accordingly. The mess comes with all the places where the old book-category naming conventions are hardcoded in, instead of using {{BookCat}} or {{BOOKCATEGORY}}; for example, this will probably turn up just about every case on the project of a book-specific template whose name doesn't start with the name of the book. For each book category renamed, one will have to carefully check for pages still belonging to the old category or its subcategories, and and also check for incoming links to the old category and its subcategories; and the templates will need to either be modified to use {{BOOKCATEGORY}}, or be renamed to use the book name and use {{BookCat}}. Renaming the templates will usually be desirable but a lot more work, and is a point where semi-automated assistance might be useful but it would actually be a pretty sophisticated task for an assistant. Anyway, I've got the basic support built into {{BookCat}} and {{BOOKCATEGORY}} now, and have set up one book category with the new naming convention, to start shaking the bugs out of the system. --Pi zero (discuss • contribs) 13:07, 19 April 2017 (UTC)

I'm not sure how this is going to play with the TODO mechanism (which I've never used and don't know how works — though I imagine I'll be figuring it out now since I find myself with a need to know). --Pi zero (discuss • contribs)

The TODO thing turned out to be not difficult, btw.

As a matter of general edification, here are (recent) numbers of book categories under the old and new scheme; as best I can reconstruct, when I started the number of extant book categories was 2813, with two new ones created before I started tracking this here (I was going to have these generated automatically, but I don't trust magic word PAGESINCATEGORY, and besides, if I occasionally update this by hand it'll make a permanent record of the dates in the page's revision history):

2200 --> 637

--Pi zero (discuss • contribs) 23:23, 24 April 2017 (UTC) – 17:30, 10 July 2017 (UTC)

Here are some notes on how to do these renamings; I've been gratified to find several other Wikibookians have pitched into help, so thought I'd share these notes here:

• Moving a book category, say from Category:Garthok Narfling to Category:Book:Garthok Narfling, does not immediately cause the wiki software to move the pages from listing under the old name to listing under the new name.

• If the pages of the book use templates that automatically detect the name of the book category, then the pages themselves will (after a few seconds) report that they belong to the new category; that is, the new category is listed at the bottom of the page. However, looking at the new category, the page is not listed, and looking at the old category the page is listed. This is because the wiki software generates the list on the category pages from an internal cache that is only updated either when the page is edited, or when a template used by the page is modified, or if you wait long enough eventually the wiki software should notice the change; I think there's a message somewhere that claims the software should get around to noticing these changes within about a month.
• To do a null edit on a page, call up an edit panel on your browser, and pick the "publish" button without making any changes. That should cause the wiki software to update its cached categorization of the page. Sometimes it takes a few second for the cache to update; and occasionally I find it necessary to do a second null edit. Remember, this won't help if the page itself doesn't claim to be in the new category rather than the old one.
• There are three general-purpose templates that detect the name of the book category. The main page of a book should always use {{subjects}}. Subpages ordinarily use {{BookCat}}; occasionally they use {{BookCat|filing=deep}}. The third template that detects the book category name is {{BOOKCATEGORY}}; some pages directly or indirectly use that for categorization. There are also some books that have book-specific templates that categorize the pages that use them, and those book-specific templates should use one of the general-purpose templates.

• If the book category has subcategories, they have to be dealt with too.

• There is a loose naming convention, not always observed, that subcategories of a book category should be named by the book-category name followed by a slash and a suffixing name. For example, the canonical name for the category of templates for book Garthok Narfling would be Category:Book:Garthok Narfling/Templates. If subcats use that convention, they need to be renamed when the parent category is renamed. Admins have an advantage here because the wiki software gives them an option, when moving a page, to move all of its subpages (up to 500). If the subcat uses {{BookCat}}, it will recategorize itself automatically. Often the category page needs editing to name its book correctly, usually via {{ROOTBOOKNAME}}; for example,

This category contains templates used in the '''''[[{{ROOTBOOKNAME}}]]''''' book.
{{BookCat}}

Pages in the subcat have to be induced to be relisted from the old subcat to the new one. If the subcat doesn't use this convention, one can either modify the subcat using {{BOOKCATEGORY}}, or move the subcat to use the convention. For example, a subcat Category:Garthok Nafling stubs might be left in place, with code [[{{BOOKCATEGORY|Garthok Narfling}}]]. I usually leave most subcats in place, but usually rename template subcats and image subcats.

• Templates don't always use the naming convention Template:<bookname>/<subname>. If they don't, the template needs to be categorized using something like

[[{{BOOKCATEGORY|Garthok Narfling}}/Templates]]

Images always need treatment like this, since they never have names of the form File:<bookname>/<subname>.

• If a book has templates, it makes sense to deal with them first, since if the template should happen to need modification in the process, that would cause the wiki software to update cache of all the pages that use the template. (This rarely works, but once e.g. I was able to avoid null-editing pages of a thousand-page book one-at-a-time by tweaking a template that was used on almost all of the thousand pages.)

--Pi zero (discuss • contribs) 23:48, 22 June 2017 (UTC)