Tag Management/Wrangling Tool w/ Stats

Feature Requests
, ,
1

WHAT DO YOU RECOMMEND?

When “tag-wrangling”, you would merge overly-similar tags, split ones that are too vague, and delete ones that aren’t useful.

To accomplish this, there has to be a way to discriminate between useful, similar, vague, and underused tags.

There’s currently no convenient tool for this task. Currently, I have to pull up each tag via a set, then filter for the tags I want to evaluate with my Mk-I Brain—quite a tedious process—and so I’d like to recommend a dedicated tag-wrangling tool that displays useful statistics about each tag and facilitates merge/split/delete operations.

(Unfortunately, as I’m not a UX guy, I will not be recommending any specifics of the tool’s UX. This is mostly about the metrics the tool will use.)

HOW COULD IT BE DONE?

The ff. discussion is structured to go from workflow needs to functional details.

Workflow Needs

Following is a brief discussion of scenarios that might lead the user to decide:

  1. Which tags to merge
  2. Which to split
  3. Which to delete

and thus what would be demanded of the hypothetical tag-wrangling tool.

(For our purposes, “merging” is equivalent to the deletion of an arbitrary selection of tags and the subsequent insertion of a single tag across all objects where at least one of those deleted tags existed.)

Merging

Someone may decide to merge two tags if they occur together all the time; if they co-occurred all the time, they might as well just be one tag, no?

Thus, the tool must include a way to measure co-occurrence between tags.

Splitting

Someone may decide to split a tag if it covers such a broad range of objects that it doesn’t help you narrow anything down.

Thus, the tool must include a way to measure a tag’s specificity.

Deleting

Someone may consider deleting a tag if it just applies to one or two objects, potentially making those objects better-served by being plainly linked to rather than tagged at all, or if it applies to almost everything in the space, but cannot be usefully split.

In this case, the tag’s specificity can be reused to diagnose these issues.

Tool Metrics

Summarizing the earlier discussion, the tool must:

  • Have a way to measure co-occurrence of tags
  • Have a way to measure specificity

The tag-wrangling tool, I imagine, takes a selection of tags and compares them against each other, coming up with certain statistics to help the user make their tag-wrangling decisions.

So, “co-occurrence” and “specificity” are the metrics of the day.

Measuring Co-occurrence

Divide the number of objects where two tags occur in common, by the union of all objects covered by both tags (i.e. no duplicates). In set theory, this is the intersection.

By the nature of the math, this is a measure of relative frequency. As such, it’s possible to implement an automatic user warning in case “co-occurrence exceeds 0.8” or some other heuristic.

Through this metric, the user can decide whether to merge or let two tags be.

Measuring Specificity: Inverse Document Frequency

The usefulness of a tag is based not only on how often it’s applied, but also on its usefulness on filtering objects; it shouldn’t occur too often (approaching a situation where filtering just gives you the entire space) nor too rarely (where filtering doesn’t give you much at all). In other words, a tag’s usefulness is related to balancing how specific its returned results are: specificity.

Thus, it makes sense to show the relative frequency of a tag’s occurrence, or in other words, the % of objects the tag applies to.

The naive metric of specificity is thus the totality of all objects, divided by the number of objects with some tag (plus one to avoid zero division errors). The higher this number, the more specific a tag.

Unfortunately, this formula will give wild results, making it hard to set numerical comparison filters.

For example, in spaces of 20,000 vs 10,000 objects, if only one object is tagged with ExampleTag, the tag will have a specificities of:

Objects-Total Objects-Tagged Specificity-Naive
20,000 1 10,000
10,000 1 5,000

(Note: the denominator has a plus one).

Ideally, the specificities should be more similar to better express the fact that it’s just one object that’s been tagged. This way, when you set a filter that depends on comparing specificity against some user-defined comparand, you wouldn’t have to change the comparand every time the space’s number of objects changes.

Thankfully, it’s possible to approach this ideal using a statistic called IDF, or “Inverse Document Frequency.” Used in information science and natural language processing, it’s calculated the same way, but churned through a log (base-10) function as the final step. This has the effect of keeping the metric stable whether you’re using large or small numbers.

Comparing with the earlier example:

Objects-Total Objects-Tagged Specificity-Naive Specificity-IDF
20,000 1 10,000 4.000
10,000 1 5,000 3.698
1,000 1 500 2.698

Of course, the values are still moving—just not as much, thus they’re a lot easier to compare and can go on a graph without flying off the screen.

These metrics will facilitate merge/split/delete operations for tags. I can’t really say much about how to put these metrics and operations together into a nice UI, but the idea of a dedicated mode or screen for it is enticing for me. Even a live table with a couple of options and columns to list tags and their stats, then some batch-editing slapped on, would be enough.

REAL WORLD USE CASES

  • Finding underused tags
  • Grouping and merging similar tags
  • Purging overly-specific or overly-broad tags

With these three powers combined, 100 tags shall become 20.

RECOMMENDED ALTERNATIVES

One could set simple warnings to show if a tag is used in less than N objects. This will easily solve tag under-usage problems.

On the other hand, it’s difficult to say in absolute terms if you’ve used a tag too much, which is why relative frequencies are more useful.

ADDITIONAL CONTEXT

In Anytype:

Outside Anytype:

  • I actually first encountered the IDF metric as used for tags in Zettlr, a Markdown-based local-only open source PKM and writing tool. I just use it for writing, though.
  • Obsidian has a tag wrangler plugin, but it does not feature stats like what I mentioned.