See a typo? Select the passage and press Ctrl+Enter.

Stagsi Cookbook

This comprehensive document is a hacker’s guide to Stagsi – an Interface to Soletude’s Tagging System.

With Stagsi you and your team can organize data with a hierarchy of tags. While this is not new (see Alternatives), the openness and flexibility of the data model, QueryLanguage, applicability to any file format, and durability (fault tolerance and number of managed objects) are distinguishing Stagsi even from the industry leaders (Adobe Bridge and Apple iTunes).

Stagsi is a commertial product but its Free edition can be used in non-commercial environments with a few light limitations, and a Professional license costs pennies compared to others and is irrevocable and non-expiring by design.

Data File Structure

Stagsi is using simple, easily accessible data storage mechanisms. At its core lies Stagsi.sqlite – a standard SQLite version 3 database which is described in detail in its own section.

Other files and folders that may appear:

Subfolders in this, Temp, Attachments and other subdirectories are used to work around certain file systems (especially FAT) limitations on big number of files that become increasingly slow when a directory contains more than a few thousand of entries. Stagsi is designed to work with hundreds of thousands of objects (each object having multiple files).

If you wish to store custom data within a Stagsi database directory, prefix your (root) files and folder with an underscore: _Docs\License.txt or just _License.txt if it’s directly under the data path. Such names are guaranteed to never conflict with Stagsi’s own data.

Welcome Page

If the database folder contains Welcome.html file, it’s presented to the user whenever it’s opened in Stagsi. Because it’s a local file, you can use the file:// scheme and reference resources (<img> and others) relatively to the database folder. However, interaction with remote resources and <script> (JavaScript) are subject to standard Internet Explorer and Windows restrictions. Basic formatting with local images and links (<a>) should always work but remember that Internet Explorer is fairly old and doesn’t support many modern web standards.

If you’re redistributing your database, you can put a license text here, or information about and links to your company or team.

You cannot directly launch processes from this page but you can reference executable files like <a href="file://run_me.bat"> and when the user clicks on such a link he’ll be asked if he wants to run it. This can be used to open tabs in Stagsi (Stagsi.exe /search "the | query"), run maintenance tasks (/rethumb), run arbitrary programs (Notepad.exe), etc.

Skeleton

Skeleton appears under Stagsi’s installation directory and holds initialization files for new databases (done when “opening” an empty directory by copying Skeleton files there). It is used to specify new database defaults.

If Skeleton\Temp\Bootstrap.bat exists then it’s ran within CommonScriptEnv in the end of initialization; Stagsi waits until it exits (ignoring its exit code) and proceeds on loading the new database. To cancel database creation use taskkill /F /PID %pid%.

Example of things you can do thanks to this folder:

Temp

May include such files, grouped into arbitrary subfolders:

If Stagsi is not running, it’s safe to remove this entire folder or any files inside it to reclaim space.

Numbered Directories

Each numbered directory includes one or more groups of files belonging to one particular imported object (e.g. an image, a video or a document). The directory in which a group is stored is determined by the object’s RowId.

Note: RowId is unique. Stagsi (or, rather, its SQLite backend) guarantees that once a RowId was allocated (by importing a new object), it will never be reused, not even if the object is deleted and another is imported (a gap will be present, where the old object’s RowId is left unused forever within this database).

Let’s assume that:

In this case, all files of this group (of this imported object) are stored in the directory named 12 – the result of rounding down the result of dividing RowId by 1000 (the limit).

Such files may exist for this group:

Thumbnail Files

There are 3 possible thumbnail files per each imported object which differ in the prefix (first letter): t, u and c.

t stands for s"t"andard thumbnail. Stagsi generates this file from the file data for known file formats (like images). For unknown formats, e.g. for MP3 audio or XLS spreadsheet, this file is not created.

u stands for “u”ser thumbnail. Stagsi doesn’t create or change this file except via the Edit Thumbnail dialog. It doesn’t exist for objects using standard thumbnails (t).

c stands for “c”ropped thumbnail. This is a smaller version of either t or u. It’s managed by the user via the Edit Thumbnail dialog. It doesn’t exist for objects using full-size thumbnails (either t or u).

This table summarizes the differences of each prefix:

Prefix Can be manually changed? When used? Vital for backups?
t No – change u or c instead. If u and c don’t exist. No, generated from file data with /rethumb.
u Yes, when Stagsi is not running.

If both u and c exist, make sure to change both of them accordingly to avoid discrepancies later.

If c doesn’t exist. Yes.
c Always, if exists. No, generated from c or u with /rethumb.

By default, when files are imported in Link mode, they reference original files by the absolute path. This is fine if the database is used on the same system or its location often changes (while location of the original files doesn’t). However, it’s troublesome if the is being used on different systems (e.g. via a shared folder or git repository), or original files are often moved.

Let’s imagine you have a folder with books and a Stagsi database which provides a “view” into that folder (book files are not inside Stagsi but can be searched and opened from there). We need the book folder and the database to be “portable” – usable from a USB stick on different systems (even without Stagsi installed), or when shared via Dropbox or GitHub, etc. which is impossible with absolute paths (the default) since files end up in different locations.

If you are sharing the book files as well as the database, you can place the database in the root folder with the books and change all link data files from C:\Users\MyBooks\... to ..\ (so that file data is searched 1 level above Stagsi.sqlite):

_Stagsi\ Bios\ Computers\ Fantasy\ History\ ...

If you are sharing the database only and your users have the book files somewhere, you can use what is called “environment variables”. These are basically arbitrary strings with a predetermined name, set via Control Panel > System > Advanced settings > Environment Variables. Create one named STAGSI_BOOKS, set it to C:\Users\MyBooks (and ask other users to do the same) and replace link data files from C:\Users\MyBooks\... to %STAGSI_BOOKS%\....

Normal object data is stored in files like 123.gif but if an object was imported in “linked” mode, the extension is omitted: 123 and the file stores just a plain text absolute path to the “real” object file (outside of the database directory).

To fix a single link, it’s usually more convenient to simply re-import the file whose link is broken – Stagsi will detect it as a duplicate and do nothing except fixing the link to point to the new location of that file.

To fix links in a more complex way, these files can be edited by hand (if Stagsi is not running). For example, if you have moved a folder to which multiple links exist, you can run a Find & Replace on this directory using an editor like Notepad++. If the old folder’s location was C:\MyDocuments and new location is F:\My then here is how it can be fixed:

On a big database, this can take longer than desired because we’re matching all files within the database. To speed it up, we can pre-filter files by wildcards. In Notepad++, setting the Filters input to the following:

?;??;???;????;?????;??????

…will restrict the search to files which names are exactly 1, 2, 3, 4, 5 or 6 characters long. This still includes some files with extensions like 12.gif but not more than first 99 RowId’s (if using traditional 3-character long extensions), leading to a significant performance benefit.

In a program that allows exclusion by wildcard (like WinRAR), specifying *.* is better as it skips all files that have an extension regardless of their name’s length.

If you a scripting guy, a little bit of Perl can make anybody’s life easier. Here’s code that replaces absolute path references (produced by the import in non-link mode) with relative:

Misc/Unprefix link paths.pl • Skip this example (#exQW) • Browse on GitHub • Adjustments (2)

chdir q[C:\My\Database]; @files = glob "*\\*"; foreach my $file (@files) { if ($file =~ /\\\d+$/) { open(my $in, $file) or die $!; my $content = <$in>; # .\ is superfluous, could be just an empty part. $content =~ s/^C:\\My\\Database\\/.\\/; open(my $out, '>', $file) or die $!; print $out $content; } }

Attachments

Stagsi allows adding extra files and folders per each database object. For example, for a video lesson you can keep notes in a simple text file. Such extras are placed in separate directories per object within Attachments using the same numbering convention as the main directory. For example, object with RowId of 12345 will use Attachments\12\12345\.

Object directory contents are entirely user-dictated and may contain any kind of files and even subfolders. Stagsi does not track or hash them.

Common Script Environment

When Stagsi starts new processes it typically sets their environment as follows:

Tools

Many Stagsi-related tasks can be automated with scripts or small programs. The Tools directory and the corresponding main menu command allow calling them easily from within Stagsi. You can write your own scripts or find a bunch of helpers created by the community at go.soletude.ca/stagsi/recipe.

If you access a certain tool often, create a hotkey by putting a & before a symbol in its file name, for example: Clean T&emp.bat – then call it by pressing Alt+T (opens the Tools menu) followed by Alt+E (Alt + symbol after the &).

Tools’ processes are started within CommonScriptEnv. For files starting with Selection (e.g. Selection to iTunes.bat) and when Object Browser’s selection is non-empty, an environment variable named SELECTION is set to a path to a text file containing information about selected object(s) in the same JSON format as Object Browser’s Copy command. Tools using this file for long (if another tool may be called before this tool exits) should copy or move this file elsewhere and work on that copy, otherwise it may change while the tool is running.

Scale and Convert Selection to PNG

Suppose you have a database of icons for mobile apps and, naturally, they are all vectors (SVGs). Stagsi can display their thumbnails thanks to its plugins but when you want to use such an icon in a program that doesn’t support SVG you have to convert it into PNG and doing it by hand is tedious.

Below is a PHP script that’s using ImageMagick to convert images selected in Stagsi into PNGs of certain square dimensions, preserving their transparency and trimming transparent background on the edges (-trim). Copy/paste it and change $size to create multiple “tools”, each converting to different dimensions (e.g. 16x16 and 32x32):

Tools/Selection to &16x16 PNG.php • Skip this example (#exVA) • Browse on GitHub • Adjustments (2)

<?php $size = 16; $imagick = "C:\\Program Files\\ImageMagick\\convert.exe"; strtok(file_get_contents(getenv('SELECTION')), '{'); $data = json_decode('{'.strtok(null)); foreach ($data->Objects as $obj) { system(escapeshellarg($imagick). " -size {$size}x$size -background none -trim". ' '.escapeshellarg($obj->FilePath).' '. ' '.escapeshellarg(getenv('USERPROFILE').'\\Desktop\\'.$obj->Title." {$size}x$size.png")); }

Backup Tools

Nobody wants to invest their time managing thousands of items if there is no clear backup strategy (this is one problem with proprietary software). Stagsi is completely open on how it stores the data making any conventional tool perfectly adequate for the task.

Read What to Backup? to learn which files you have to store and which can be safely left out.

As for the tools, here are some options:

Utility Class Tools Pros Cons
Archivers WinRAR, 7-Zip, etc. Fast (literally 1 click if using WinRAR profiles).

Extremely simple to use.

Encryption and corruption protection (WinRAR) out of the box.

Subsequent backups are as large as the entire database which is a big problem for frequent backups.
Version Control Systems SVN, git, Mercurial, etc. Drastically reduced storage requirements.

Some are distributed (can be used as a “shared folder”).

Steeper learning curve.

Slow commits with tens of thousands of files.

Incremental implies slow restoration and larger impact of a history corruption.

Cloud ownCloud, Dropbox, G.Drive, AWS, etc. Effortless and simple to use.

Distributed and (reputedly) reliable.

May be paid.

May be third party.

Not versioned – can’t revert to an “earlier version” if your folder was synced after corruption.

Backups dar, tarsnap, Acronis True Image, VSS, etc. Pros and cons are very specific to a particular product. For instance, cross-platform open-source dar allows incremental backups with encryption, volume support, scripting, etc. but it aimed at power users. Acronis True Image is the opposite, more like a “backup cloud” for home users.

What to Backup?

With Stagsi, you don’t need to back up the entire data directory because some data can be reconstructed from another. However, omitting wrong files may lead to small but unrecoverable database.

Short and safe answer: use a tool like WinRAR or git that supports excluding files by wildcards and specify these:

Temp/* */t*.* */c*.*

In other words, omit the Temp directory (DataTemp) and files in the numbered folders starting with t or c. After extracting a backup made this way, run the /rethumb command and you are ready to go.

With WinRAR, you can make backups significantly faster if you add the following to the list of files added without compression (since they’re already compressed by nature):

*.gif *.png *.jpg

WinRAR also supports storing settings under profiles – prepare it once and back up with a single click.

Long answer: the following files are of interest, in order of importance:

And the following files are not important:

SQLite

What is SQLite?

SQLite is a database engine used by world-popular projects like Firefox and Android. Stagsi is using SQLite version 3 as a backend for storing metadata (image properties, tag hierarchy, etc.). Advanced users can manipulate this database directly using the standard tools.

SQLite Browser for Windows, Linux and Mac

Homepage: sqlitebrowser.org.

This is a free, easy to use program for all popular operating systems.

Let’s imagine you have a large number of objects with a typo in the title: “gmae” instead of “game”. You can fix this by issuing this SQL query (note that it’s case-sensitive – “Gmae” isn’t affected):

UPDATE Objects SET Title = Replace(Title, "gmae", "game")

Or suppose you want to wrap titles of all tags’ of some parent into brackets. For this, first determine the parent tag’s RowId by hovering over it in Stagsi’s tag dialog, then utilize the SQL’s concatenation operator (||):

UPDATE Tags SET Title = '[' || Title || ']' WHERE ParentRowId = 12

SQLite Firefox Addons

If you have Firefox installed, then you can avoid installing any other software for simple SQLite tasks.

Usage is similar to SQLite Browser. For example, you can multiply the Weight of all objects’ tags’ by 10 with this SQL query:

UPDATE ObjectTags SET Weight = Weight * 10

sqlite3 Command-Line Tool

The SQLite project provides a cross-platform console tool for running queries on SQLite databases. It’s great for use in shell scripts.

Here’s a query that returns a comma-separated list of RowId’s of objects that are over 10 MiB in size:

SELECT GROUP_CONCAT(RowId) FROM Objects WHERE FileSize > 10485760

Reading from PHP

If you are a programmer, you can automate many tasks with PHP (or similar languages). PHP comes with built-in support of SQLite 3 via PDO.

Here is a script to update hashes of the specific objects only (handles linked files; doesn’t handle large files due to a different hashing mechanism):

Database/rehash simple.php • Skip this example (#exJV) • Browse on GitHub • Adjustments (3)

<?php $stagsiDataPath = "C:/My/db"; $db = new PDO("sqlite:$stagsiDataPath/Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $getStmt = $db->prepare("SELECT * FROM Objects WHERE RowId IN (1, 2, 3)"); $getStmt->execute(); $rows = $getStmt->fetchAll(PDO::FETCH_OBJ); $setStmt = $db->prepare("UPDATE Objects SET Hash = ? WHERE RowId = ?"); foreach ($rows as $row) { $base = "$stagsiDataPath/".floor($row->RowId / 1000)."/$row->RowId"; $file = is_file($base) ? file_get_contents($base) : "$base.$row->Format"; $setStmt->bindValue(1, md5_file($file)); $setStmt->bindValue(2, $row->RowId); $setStmt->execute(); }

Comparing the NULLs

Most people unfamiliar with SQL make a common mistake of assuming that WHERE Column = NULL will match if Column is NULL. However, in SQL nothing matches a NULL, not even another NULL so WHERE NULL = NULL will never match. Instead, for comparing with NULL use the IS operator: WHERE Column IS NULL or Column IS NOT NULL (not <>).

For example, to convert names of all your tags to lower case except for system tags like Last Imported: UPDATE Tags SET Title = LOWER(Title) WHERE System IS NOT NULL

Note: LOWER() only works with Latin symbols and will corrupt strings containing other characters.

Tags

System Tags

System tags are special kind of tags maintained by Stagsi. Unlike normal tags, they:

However, just like normal tags, they:

The following system tags exist:

Automatic Localization

Actual titles for system tags having blank Title in the database are taken from the current localization – this way you and that guy in Russia see the Deleted tag differently even if sharing the same database. However, because the localized title is used in the search, queries now become localization-specific and on your system Удалённые will give an error.

If this is a problem for you – give system tag(s) an explicit name (even if it matches your localization exactly, e.g. “Deleted”). Title will stop being an empty string and Stagsi will treat it literally.

Volume Indicator

In most cases, when a tag’s name is displayed it will be accompanied by a “volume indicator” – a small vertical “progress bar” indicating how many objects in a database are tagged with this tag using a logarithmic scale.

For example, if your database has 20 objects in total and 5 of them are tagged with tag A, then A’s volume is log10(5+1) / log10(20+1) = 58.8%.

Weight & Relevance

An object has not just tags but also relevance of that tag to this particular object – its “weight”.

Imagine you have 2 images: a portrait of someone and a room with a hanging portrait on the wall. In the first case, the “portrait” tag is much stronger, more relevant to the description of the image than in the second, where it’s a mere detail, not the main theme.

Yet in most traditional tagging systems, you get a binary “have tag”/"have no tag" relationships and you have to skip describing small details or use different tags for them (e.g. “portrait_image” and “hanging_portrait”) to avoid polluting the search with irrelevant results. Or (as in Adobe Bridge) you get a 5-star rating system but it’s global and doesn’t tell us if the picture is “generally great” (5 stars) or the “portrait frame is excellent” (same 5 stars).

In Stagsi, you can still use one tag for both images, but assign +1 weight to the first (a portrait) and 0 to the second (a handing picture). And the equivalent of “generally great” (a generic rating) is just a weight of some specific tag, like “greatness”.

Weights:

Filtering the Tree

If you have over a hundred of tags you will find that the Quick filter input is the most convenient way to browse your tag hierarchy. Here are some helpful tips regarding both the tags’ and saved searches’ trees:

Tag Decks & Deck Sets

One unique feature of Stagsi (among many others) is a large array of tagging hotkeys – over 2 dozens.

Hotkeys are bound to tag slots, which can be of two kinds: global and deck-specific. Each slot may be assigned one or more tags that are applied to objects in the current selection or under the mouse cursor (if you’ve enabled this option).

First, you have 5 global tag slots that respond to Q W E R T keys (without modifiers). Using Shift modifier would toggle tags instead of adding only. Another 5 history slots (Y U I O P) are like global but get automatically filled as you pick tags elsewhere.

Second, you have 10 deck-specific tag slots that respond to numeric keys (1 2 3 … 0), again without modifiers or with Shift.

Then you have 10 decks that are switched with Alt+numeric_key. So you have about 100 tag slots (each with multiple tags!) that are reachable by at most 2 keystrokes (switch deck, then apply tags).

Finally, if that was not enough, you can switch decks themselves by means of a button with a menu (no hotkey this time). The menu allows you to create as many tag decks (organized into deck sets) as you wish. So, with 2 clicks and 2 keystrokes at most you can reach pretty much every set of tags imaginable.

Now, a practical example.

Global slots are independent of decks, and they’re also few. Fill them with very common, generic tags that you’ll likely have, such as: “tag_me” (for revisiting this object’s tag list later), “may be junk” (stuff that could be deleted), “irrelevant” (secondary images that are details of some others, to avoid cluttering search results), “ecclesia project” (what you’re currently busy with), etc.

History slots are great for rarely used tags. First you manually pick some via the dialog, then they get assigned to these slots and you can recall them quickly, until they’re “shifted away” by more recent selections.

Deck-specific slots are to be filled with thematic tags. For example, if you’re a game designer and you are tagging animation references, then the 10 available slots may be filled with animation kinds, such as: idle, walk, run, attack, jump, looped, etc.

Once done with animations, you can switch to another deck used for describing pictured objects: cat, man, technology, furniture, nature, etc.

If your database is used for more than one kind of objects – for example, also video tutorials or phrase collection for writing stories, then you’ll quickly grow out of 10 decks by 10 slots and start using a specific deck set for a specific kind of workflow.

Mass-Merging

Imagine you have done several imports with tags extracted from file names, using a regexp so that ic_personal_video_black_36dp.png (a typical Android resource file name) got tags personal and video. However, with one of the imports you put tags under a different parent (perhaps straight into the root), as a result getting two tags: personal in the root and personal under Tags – and now you need to remove the duplicates.

This is usually solved by using Merge Into (This) commands but if you have several thousands of such tags cleaning them up by hand would be ridiculous. Worry not – you can do that in one go if you work directly on the database (with care!) since merging is basically changing the TagRowId field’s value of the ObjectTags table from the old (to-be-merged) tag to the new (combined/recipient) tag. Here’s a PHP script doing that:

Database/Mass merge.php • Skip this example (#exQU) • Browse on GitHub • Adjustments (3)

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $rootTagRowId = 1234; $db = new PDO("sqlite:C:/Stagsi DB/Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $updateStmt = $db->prepare("UPDATE ObjectTags SET TagRowId = ? WHERE TagRowId = ?"); $tags = $db->query("SELECT RowId, * FROM Tags WHERE ParentRowId = $rootTagRowId") ->fetchAll(PDO::FETCH_OBJ); $dupTags = []; foreach ($tags as $tag) { $title = preg_replace('/^:+/u', '', $tag->Title); $ref = &$dupTags[mb_strtolower($title, 'utf-8')]; $ref[] = $tag; } foreach ($dupTags as $tags) { if (count($tags) > 1) { $main = $tags[0]; foreach ($tags as $tag) { if ($tag !== $main) { echo "merge $tag->RowId into $main->RowId", PHP_EOL; $updateStmt->bindValue(1, $main->RowId); $updateStmt->bindValue(2, $tag->RowId); $updateStmt->execute(); } } } }

To use it:

“Smart” Collections

Some programs (like Adobe Bridge and Apple iTunes) have two kinds of collections: “regular” and “smart”. The first do not automatically refresh when keywords are updated while the second do.

Stagsi doesn’t have “collections” per se. The closest analogy of “smart” ones is saved searches: whenever you repeat a search (saved or not – saving is just a faster way to enter the search expression) only actual image tags are checked, so searches are never out of date.

“Regular” collections in Stagsi are a special case of “smart” – create a tag like “My Collection 01” and assign it to the images of interest, then search for this tag alone.

Filter by Path

One common question of new Stagsi users is: how to find images that once were on some file system path?

Short answer: not possible.

Long answer: paths are remnants of the traditional file & folder hierarchy mindset. If you’re trying Stagsi, it means you are not satisfied with that, and for a good reason. Stagsi doesn’t work with paths.

Now, of course there is an “indirect” way. What are file paths and names? They’re the kind of tags, if you think about it. Your birthday photos may be in C:\Users\Eric\Junk\2018party – and the last part, “2018party”, is what identifies the collection. And you want to filter based on that – just as if it were a tag. So the real question is: can you convert path components into tags?

And yes, that is possible. The Import dialog offers tagging by wildcard and tagging by regular expressions (regexps). For example, *\2018party\* wildcard and \\2018party\\ regexp can both solve the birthday problem. Read those sections for more details.

Improving Relevance

Some users may find that highly relevant objects are lost within many not that relevant results. Some tips that may help:

Query Language

A good deal of the Stagsi’s versatility comes from its search query language. It’s not complex, and by mastering it you will master Stagsi itself.

In the following explanation we are assuming this tag hierarchy:

+ people + male + elderly + female + nature + animals + cat + bird + landscape + winter

A search query consists of atoms – tags references. They can be in short and long forms. For the cat tag, the short form is just that – “cat”, and the long form is: “nature/animals/cat”, i.e. the combination of all parents’ titles.

The short form can be used if the tag’s own title is unique within the entire database. If we had another cat tag (under another parent tag) – we could not use this form, otherwise there would be no telling which of the cat tags we wanted to search for. But if another cat had no parents (was in the root), then writing just “cat” would refer to that root cat (since “cat” is both its short and long forms).

When you put two atoms – tags – next to each other (separating with a space), you are searching for both of them. To locate all images of cats on a winter landscape you do: “cat winter”.

If you separate the tags with a pipe symbol rather than a space, you are searching for either of them. To locate images for all people, you can do: “male | female”.

However, there’s a better way. If you prefix a tag with a tilde, you look for that tag as well as any of its child tags at once: “~people”. This is just a shorter form of writing “people | male | elderly | female”. As you’d guess, to locate all animals you’d do “~animals” rather than enumerating every type of animal by hand. This works on any depth so that “~nature” includes all animal’s as well as landscape and winter.

Prefixing a tag with a dash “blacklists” it – objects with this tag are omitted from the results, even if they have some other tags that match the query. To look for non-winter cats you do: “cat -winter”, and to look for all non-winter animals you do: “~animals -winter”. You can even blacklist tags that are matched by the tilde: “people -elderly” gives you all people (male and female) who are not also elderly.

Prefixes can be combined. To look for all landscapes without any animals you do: “landscape -~animal”.

But wait, there’s more. You can group tags with brackets. If you want all landscapes without animals, unless they are a winter landscape – you do: “landscape (winter | -~animal)”. This reads as: “take an image if it has landscape and either has winter or has no animal tag or one of its sub-tags” – i.e. “either (landscape and winter) or (landscape and no animal and no animal sub-tag)”.

Finally, in the same situation you could use a tag, you can refer to a particular object using its RowId – just prefix it with /. This form is indispensible if you have pre-filtered objects (e.g. an external search tool using SQLite) and you only need to display them.

Other notes:

Cheatsheet

short_form long/form "remember/to quote" "long/form"/quoted us all three either | of | us ~me_and_children -blacklist -~not_me_or_children me_plus (either | -us) and (/123 | /456) "id"/s

Parsing Libraries

The repository with samples contains a very precise AST parser of Stagsi’s query syntax in multiple languages, including PHP, JavaScript, C# and plain C. Feel free to use it in your projects (public domain CC0 license).

If you plan on extending the query language with your own syntax, consider avoiding potential clashes in the future with the standard syntax by making sure your terms start with a blank title and have either 3 or more titles or 2 titles, of which first is non-numeric (doesn’t consist entirely of 0-9 symbols).

For example, if you want to allow referring to tags by their RowId, it’s safe to use /tag/123 or /123/tag tag but not /123 or tag/123.

Custom Sorting with Weights

Let us remember that tags may be also used for sorting, not just searching. The technique below allows practically unlimited “sorting profiles”.

For example, you want images to be sorted by brightness or another aspect that is not exactly a “material tag” like “flower” or “sun” (even though brightness can be explained as “very bright”, “a bit not too bright”, etc. – using numeric qualifiers is much more intuitive). For this, create a tag – “brightness”, and add it to all images but with a different Weight. This doesn’t make images immediately sorted – you need to add “brightness” to your search query and make sure the relevance sorting is enabled (it is by default).

Custom Sorting with Random

Let’s imagine you have a bunch of week day-themed objects and you want to see ones that correspond to the today’s date first. It may sound bizarre (call it “flexible thinking”), but most script-friendly way to achieve this is with the Random column.

How does Stagsi produce random sorting? It’s simple: every object in the database gets a random number when imported (and when Randomize or /rerand commands are used) which is used to compare it with others. Bigger Random – later sorting (when in ascending mode).

Nothing prevents us from hijacking this field and using for our purpose, temporary or not, for all database objects or only some. Another advantage is that we can “undo” our sorting by simple randomization.

Note: it’s recommended to keep Random within this range: -2147483647 to 2147483647.

In this example, we’re assuming you have a tag named “Weekdays” which has 7 child tags, each starting with a digit where 0 signifies Sunday (the British, rejoice) and 6 signifies Saturday. Let’s throw in some Powershell and SQL (using sqlite3.exe from sqlite.org):

Misc/Sort by weekday.ps1 • Skip this example (#exCQ) • Browse on GitHub • Adjustments (1)

$db = "F:\Stagsi\DB\Stagsi.sqlite" $day = (Get-Date).DayOfWeek.Value__ # First, reset order of all objects. sqlite3 "$db" "UPDATE Objects SET Random = 0 WHERE RowId IN ( SELECT o.RowId FROM Objects o JOIN ObjectTags ot ON ot.ObjectRowId = o.RowId WHERE ot.TagRowId IN ( SELECT RowId FROM Tags WHERE ParentRowId = (SELECT RowId FROM Tags WHERE Title = 'Weekdays') ) )" # Then, bump objects with the matching weekday. sqlite3 "$db" "UPDATE Objects SET Random = -100 WHERE RowId IN ( SELECT o.RowId FROM Objects o JOIN ObjectTags ot ON ot.ObjectRowId = o.RowId WHERE ot.TagRowId IN ( SELECT RowId FROM Tags WHERE ParentRowId = (SELECT RowId FROM Tags WHERE Title = 'Weekdays') AND Title LIKE '$day%' ) )"

Custom Sorting with C# Plugin

Since Stagsi version 1802, IPluginService exposes a new method (see Plugins for general information):

void RegisterImageSorterV1(Func<List<IImageSortingV1>, List<IImageSortingV1>> Sorter, ImageSorterV1)

Plugin-based sorting works in two phases: generation (once per each object in the database, potentially a lengthy process) and sorting (once per search, quick). Generator returns arbitrary data that is saved in the database under Sorting\Plugin.Class.Name\Group\RowID (Group = RowId divided by FolderSize). Sorting receives that data and may use it to reorder user’s search results.

IImageSorterV1 fields:

IImageSortingV1 describes each search result and has these fields:

Sorter may modify the List<> it receives and may return the same object or a new one. If it deletes some members, they’re re-added to the end (ordered by RowId). If it adds new members, they’re ignored.

Randomize as a Marker

Even though the Random column may be signed, Stagsi itself only generates positive numbers when commands like Randomize and /rerand are used. This fact can be utilized to implement a “have seen” or another similar “yes/no” marker:

With this approach you cannot directly search for “marked” objects from within Stagsi but at least you don’t have to close it to retrieve marked object RowId’s.

Thumbnails

Editing Thumbnails

Stagsi can generate thumbnails for only a handful of formats, mostly image-based (but you can write your own plugins). For others, none are generated and a placeholder image is used in the browser.

However, you can manually add thumbnails – and the dialog for doing so has numerous optimizations to make the task easy:

Many Objects, Same Thumbnail

If you have imported 24 videos (of a single TV series) and Stagsi cannot generate thumbnails for this format, you end up with no thumbnails. But you want all these videos to have the same thumbnail. One option is to repeatedly set the same thumbnail via the Edit Thumbnail dialog. Another option is to paste user thumbnails directly into the data directory using a 3rd party batch renaming tool (ReNamer used here):

Power users may go further and utilize NTFS hardlinks to avoid wasting space for each copy as well as propagating changes done to one thumbnail to another. There is a convenient Windows Explorer shell extension schinagl.priv.at/nt/hardlinkshellext… that allows working with hard- and soft-links (symbolic links) like with regular shortcut files (.lnk), supporting drag & drop and other commands.

Another option suitable for scripting is the built-in fsutil command:

fsutil hardlink create C:\new.txt C:\existing.txt

The above command can be used in a script with a loop, or we can use Notepad 2e:

Batch Cropping

Stagsi stores crop state in the database as 4 floating-point numbers – relative distance from the top-left corner. For example, cropped top-right corner of the image has this state:

Let’s suppose you have a large number of portraits of standing people. Because of too different aspect ratio, they might not be convenient to browse at small zoom (with small thumbnails). You can batch-crop all such thumbnails to only display the top third’s horizontal center, i.e. X = 0.25, Y = 0, Width = 0.5, Height = 0.33.

We will set these crop settings using SQL (SQLite), but we also need to tell Stagsi that our objects need cropping – this is determined by the existence of c files. If all of your objects of interested already have cropped thumbnails, then some steps can be skipped, otherwise their stub files will need to be created.

Permanent Cropping

If you’re making a database of computer programs (or games) then you might be assigning screenshots (made with PrintScreen) to each object (a program or a game) and cropping them using Edit Thumbnail dialog. This works but full-size screenshots remain on disk as u-files (see ThumbU).

If you don’t plan on ever changing the crop region, you can free up the space by making the cropped version permanent (i.e. “full-size”) by:

Here is a helper PHP script:

Thumbnails/Permanent crop.php • Skip this example (#exCA) • Browse on GitHub • Adjustments (3)

<?php $rowidsFile = 'c:/rowids.txt'; $dataPath = 'c:/stagsi/db'; $ids = []; foreach (file($rowidsFile) as $id) { if ($id = trim($id)) { $path = $dataPath.'/'.floor($id / 1000).'/'; rename("$path/c$id.png", "$path/u$id.png"); $ids[] = $id; } } echo 'UPDATE Objects SET ThumbCropX = 0.0, ThumbCropY = 0.0, ThumbCropWidth = 0.0, ThumbCropHeight = 0.0 WHERE RowId IN (', join(',', $ids), ')';

It takes RowId’s of affected (cropped) objects from a text file ($rowidsFile), performs step 1 above (renames c into u) and outputs a SQL statement that you should run against that database using SQLite Browser or command-line sqlite3.exe (this is step 2).

Stagsi must not be running while carrying this out.

Batch Transformation

Let’s imagine we want to make some of our thumbnails grayscale. This can be done with ImageMagick and cmd.exe kun-fu.

Here’s a batch file that converts all standard thumbnails (of formats that Stagsi recognizes):

Thumbnails/Grayscale using ImageMagick.bat • Skip this example (#exNX) • Browse on GitHub • Adjustments (1)

setlocal ENABLEDELAYEDEXPANSION cd /D C:\...\Database for /R %%i in (t*.*) do ( set F=%%i convert "%%i" -set colorspace Gray -separate -average "u!F:~1!" )

Just close Stagsi and run this batch file (replacing the database path accordingly).

Text Overlay with Ruby

Often a good idea is to annotate a thumbnail with some short caption. The below Ruby script (using RMagick, a Ruby interface to ImageMagick; rmagick.github.io) processes a file in a simple which could be produced by a spreadsheet program like MS Excel or by hand, where each line is a text to be drawn, starting with a numerical object’s RowId followed by a space, comma or semicolon. For example:

13, Cool GFS 15 FF XII video 87; Sketching guide

Here’s the script that etches the string from the string file on a thumbnail of every object in the database (note how it correctly copies the t-file if neither u nor c are found – t-files should never be edited):

Thumbnails/Text overlay.rb • Skip this example (#exHO) • Browse on GitHub • Adjustments (2)

require 'rmagick' require 'fileutils' data_path = 'C:/Stagsi/Database' IO.read('strings.csv') .scan(/^\s*(\d+)(\s*[,;]\s*|\s+)(\S.*)$/) .each { |a| rowid, delim, str = a mask = "#{data_path}/#{(rowid.to_i/1000).floor()}/[cut]#{rowid}.*" files = Dir.glob(mask) .map { |f| [File.basename(f)[0], f] } .to_h if files['c'] then file = files['c'] elsif files['u'] then file = files['u'] elsif files['t'] dir, file = File.split(files['t']) file = "#{dir}/" + file.sub(/./, 'u') FileUtils.copy files['t'], file, :verbose => true else next end im = Magick::Image.read(file).first text = Magick::Draw.new # Actual overlaying happens here. text.annotate(im, 0, 0, 0, 0, str) { self.gravity = Magick::SouthGravity self.pointsize = 50 self.fill = 'black' self.font_weight = Magick::BoldWeight self.stroke = 'white' self.stroke_width = 3 } im.write(file) im.destroy! }

Supported Formats and Plugins

Stagsi can deal with any kind of files, even those it doesn’t support. “Support” in this case means automatic thumbnail generation and meta-data extraction. For “unsupported” formats you’ll see placeholder images in the browser, which you can easily replace by hand (see that section).

The ImportUnknownFormats setting (enabled by default) controls if Stagsi skips unsupported formats during import or not.

The actual support is provided by plugins (see Plugins for writing your own). Standard plugins coming with Stagsi by default include:

More standard plugins that you can download from go.soletude.ca/stagsi/forum:

These meta-data formats are supported by means of MetadataExtractor: EXIF, IPTC, XMP.

Custom Thumbnails – Manual Generation

To “support” a format, you could write a tool to create thumbnails for data files that are missing them. Unlike writing a plugin, you can use any language (not just C#) but you will have to run this program or script by hand (it won’t respond to /rethumb).

Here’s a C/C++ program that renders file type icon for every object that has no thumbnail (t-automatic or u-user-assigned) – assuming ThumbFormat is PNG:

Thumbnails/Generate from file icon.c • Skip this example (#exEO) • Browse on GitHub

// Compile with: cl /TP this.cpp_or_.c /link gdiplus.lib shell32.lib #include <windows.h> #include <strsafe.h> #include <Gdiplus.h> bool has_thumb(wchar_t *data_path, wchar_t *dir, wchar_t *prefix, long rowid) { wchar_t mask[MAX_PATH]; StringCchPrintfW(mask, MAX_PATH, L"%s\\%s\\%s%ld.*", data_path, dir, prefix, rowid); WIN32_FIND_DATAW find = {0}; HANDLE handle = FindFirstFileW(mask, &find); if (handle == INVALID_HANDLE_VALUE) { return false; } FindClose(handle); return true; } // Procedural-style GDI+. Close to black magic. // https://forum.antichat.ru/threads/268293/ bool hicon_to_png(wchar_t *dest, HICON icon) { // http://www.masm32.com/board/index.php?topic=5782.0 // Type: image/bmp, GUID: {557CF400-1A04-11D3-9A73-0000F81EF32E} // Type: image/jpeg, GUID: {557CF401-1A04-11D3-9A73-0000F81EF32E} // Type: image/gif, GUID: {557CF402-1A04-11D3-9A73-0000F81EF32E} // Type: image/tiff, GUID: {557CF405-1A04-11D3-9A73-0000F81EF32E} // Type: image/png, GUID: {557CF406-1A04-11D3-9A73-0000F81EF32E} GUID png_format = {0x557CF406, 0x1A04, 0x11D3, 0x9A, 0x73, 0x00, 0x00, 0xF8, 0x1E, 0xF3, 0x2E}; Gdiplus::GdiplusStartupInput si = {0}; si.GdiplusVersion = 1; ULONG_PTR token; int res = Gdiplus::GdiplusStartup(&token, &si, NULL); if (!res) { Gdiplus::GpBitmap *bmp; res = Gdiplus::DllExports::GdipCreateBitmapFromHICON(icon, &bmp); if (!res) { res = Gdiplus::DllExports::GdipSaveImageToFile(bmp, dest, &png_format, NULL); Gdiplus::DllExports::GdipDisposeImage(bmp); } Gdiplus::GdiplusShutdown(token); } return !res; } int wmain(int argc, wchar_t** argv) { wchar_t *data_path = argv[1]; wchar_t dir_mask[MAX_PATH]; HRESULT o = StringCchPrintfW(dir_mask, MAX_PATH, L"%s\\*", data_path); WIN32_FIND_DATAW dir = {0}; HANDLE dir_handle = FindFirstFileW(dir_mask, &dir); if (dir_handle == INVALID_HANDLE_VALUE) { return 1; } do { wchar_t *end = dir.cFileName; while (*end && *end >= L'0' && *end <= L'9') { end++; } if (*end) { continue; } wchar_t file_mask[MAX_PATH]; StringCchPrintfW(file_mask, MAX_PATH, L"%s\\%s\\*", data_path, dir.cFileName); WIN32_FIND_DATAW file = {0}; HANDLE file_handle = FindFirstFileW(file_mask, &file); if (file_handle == INVALID_HANDLE_VALUE) { continue; } do { // strtol()/wcstol() allow leading whitespace and sign. if (*file.cFileName < L'0' || *file.cFileName > L'9') { continue; } wchar_t *end; long rowid = wcstol(file.cFileName, &end, 10); wchar_t data_file[MAX_PATH]; StringCchPrintfW(data_file, MAX_PATH, L"%s\\%s\\%s", data_path, dir.cFileName, file.cFileName); if (*end == L'.') { // Not a link file. } else if (!*end) { HANDLE link_handle = CreateFileW(data_file, GENERIC_READ, 0, NULL, OPEN_EXISTING, 0, NULL); DWORD link_read; char buf[MAX_PATH]; // MultiByteToWideChar() won't write '\0'. memset(data_file, 0, sizeof(data_file)); if (link_handle == INVALID_HANDLE_VALUE || !ReadFile(link_handle, buf, sizeof(buf) - 1, &link_read, NULL) || !link_read || !MultiByteToWideChar(CP_UTF8, 0, buf, link_read, data_file, MAX_PATH)) { continue; } CloseHandle(link_handle); } else { continue; } if (has_thumb(data_path, dir.cFileName, L"t", rowid) || has_thumb(data_path, dir.cFileName, L"u", rowid)) { continue; } SHFILEINFOW info = {0}; SHGetFileInfoW(data_file, 0, &info, sizeof(info), SHGFI_ICON | SHGFI_LARGEICON); wchar_t thumb_file[MAX_PATH]; StringCchPrintfW(thumb_file, MAX_PATH, L"%s\\%s\\u%ld.png", data_path, dir.cFileName, rowid); hicon_to_png(thumb_file, info.hIcon); } while (FindNextFileW(file_handle, &file)); FindClose(file_handle); } while (FindNextFileW(dir_handle, &dir)); FindClose(dir_handle); }

Custom Thumbnails – Via Scripts

If you have a small number of objects that need custom thumbnails, a more efficient approach is to apply them from within Stagsi on demand.

First, write a script that extracts object ID from the path it’s given (the file’s name is u123.ext where 123 is the ID – see DataNumbered), retrieves required info by that ID (e.g. the object’s main file), generates a thumbnail image and saves it to that path (argv[1]).

Second, invoke Stagsi’s thumbnail dialog with F3, then press F3 again to call an external editor. A list of programs will appear (unless you have configured it to always call a specific program) – add your script to this list. Now you can quickly call it for a particular object that needs custom thumbnails, working like specialized Tools.

Custom Thumbnails – C# Plugin

Plugins are loaded (in arbitrary order) from two folders: Plugins near Stagsi’s EXE and Plugins in the database. A plugin may be either a .dll or a .cs file. .cs is a C# source code file compiled and loaded on runtime as if it was a pre-compiled DLL, with implicit references to DLLs already loaded into Stagsi and to all DLLs in all Plugins folders, and with explicit reference from // ref: c:\path\to.dll comment(s) in the end of the file (ignoring blank lines).

Note: our community repository go.soletude.ca/stagsi/cookbook has a few plugins which show how to wrap a C++ library into a C# plugin, e.g. psd2png.

Each plugin must define one or more classes containing a static function named StagsiPlugin() returning void and accepting a single argument typed Soletude.Stagsi.Plugins.IPluginService exposing this method (version 1802 added SortingPlugins): void RegisterImageLoaderV1(Func<Stream, IImageLoadingV1, BitmapSource>, ImageLoaderV1)

ImageLoaderV1 specifies additional configuration and currently contains just one property: FileExtensions (an array of string) – typical file name extensions (without the period) for the kind of format this handler is able to process. This is used in file dialogs and internal optimizations and may be empty. First string is considered the “canonical” extension visible as a group title for other extensions (aliases) – e.g. JPG, alias: JPEG.

When Stagsi needs to decode a file into an image data it calls all registered handlers (in arbitrary order) giving them access to the file’s raw data (Stream) and its information (IImageLoadingV1). If a handler is unable to process the data it returns null.

Stream has its pointer at the beginning of the data and may or may not correspond to a local file. The handler may close it (CanRead() returning false – may happen if the stream’s handle is transferred to native code) or advance regardless if null is returned or not.

Note: regardless of FileExtensions a handler may be called for any input file (which may not be a local file or may have an invalid extension, e.g. .png instead of .gif) and so must check if the input Stream contains data that it recognizes, e.g. using “magic signatures” (see the popular file utility astron.com or this common signature repository www.garykessler.net/Library/file_sigs.h…).

IImageLoadingV1 fields:

Conventions for plugin writers (assuming a hypothetical plugin named “WebPage”):

You can debug your plugin with Visual Studio’s Attach to process command, setting a breakpoint in your plugin’s code and invoking an operation in Stagsi that calls it (e.g. Copy in Object Browser).

With all this, a sample plugin rendering text files into the thumbnail’s box (save as Plugins\TxtPlugin.cs and restart Stagsi):

Thumbnails/TxtPlugin.cs • Skip this example (#exGP) • Browse on GitHub • Adjustments (6)

using System.Globalization; using System.IO; using System.Text; using System.Windows.Media.Imaging; using System.Windows.Media; using System.Windows; namespace Soletude.Stagsi.Plugins { public class Txt { private const string TxtFormatName = "txt"; private const string EndLineChars = "\r\n"; private const string RenderFontName = "Arial"; private const int MaxLength = 10000; private const double DefaultSize = 200; private static readonly string[] InvalidStrings = {"\0"}; public static void StagsiPlugin(IPluginService service) { service.RegisterImageLoaderV1(RenderTxt, new ImageLoaderV1 { FileExtensions = new[] { TxtFormatName }}); } private static BitmapSource RenderTxt(Stream input, IImageLoadingV1 parameters) { var len = input.Length; if (input.Length > MaxLength) { len = MaxLength; } byte[] buf = new byte[len]; input.Read(buf, 0, buf.Length); string allText = Encoding.UTF8.GetString(buf); foreach (var v in InvalidStrings) { if (allText.Contains(v)) { // Binary data - skip. return null; } } allText = allText.Replace(EndLineChars, " "); var txt = new FormattedText(allText, CultureInfo.CurrentCulture, FlowDirection.LeftToRight, new Typeface(RenderFontName), 10.0, Brushes.Black); return WriteTextToImage(txt, new Point(0, 0), parameters); } public static BitmapSource WriteTextToImage(FormattedText text, Point position, IImageLoadingV1 parameters) { var wid = parameters.Width; if (wid == 0 || double.IsNaN(wid)) { wid = DefaultSize; } var hei = parameters.Height; if (hei == 0 || double.IsNaN(hei)) { hei = DefaultSize; } var visual = new DrawingVisual(); var prop = hei / wid; hei = (int) (wid * prop); text.MaxTextWidth = wid; using (var dc = visual.RenderOpen()) { dc.DrawRectangle(Brushes.White, null, new Rect(new Size(wid, hei))); dc.DrawText(text, position); } var target = new RenderTargetBitmap((int) wid, (int) hei, 96.0, 96.0, PixelFormats.Default); target.Render(visual); parameters.Format = TxtFormatName; return target; } } }

Browser Copy

This is one of those seemingly unassuming commands that you find more and more uses for every day.

When you perform a copy on a selection in the browser, the clipboard receives data of 3 types:

The first type is the main source of this command’s versatility – we’ll even call it “the Stagsi interchange format”. The string it creates looks like this:

Stagsi JSON 1 {JSON}

First three parts are constants (“1” is the format version number), acting as a marker: if a string in the clipboard begins with this sequence then it’s assumed to be “paste-able”.

The fourth part is a multi-line JSON object which mimics the database structure with a few extra fields. This object holds information about the selection and is independent of the Stagsi process that has created it (i.e. this string can be pasted to another Stagsi process after the originating Stagsi was closed).

Synthesized Paste

Often you need to import data from another tool, e.g. from an XML dump of a custom format. One way is to write to Stagsi.sqlite directly as we have already demonstrated:

Another way is using the JSON format produced by the Copy command where you produce this JSON and Stagsi consumes it by means of Paste:

Below is a script converting XML data from MyAnimeList.Net into Stagsi’s JSON format creating a temporary .url file for each object linking to that series’ page on the website. For Paste to work, the JSON must be prepended by the version info (see above, e.g. Stagsi JSON 1). You can use clip to copy a string to the clipboard from the console.

User’s score (series rating) appears both as a tag (under Score) and a Weight so that to search results by this score you just include the ~Score into your search query (and enable Relevant first).

Import/Convert MAL.php • Skip this example (#exLQ) • Browse on GitHub

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); mb_internal_encoding('UTF-8'); list($self, $file) = $argv + ['', '']; if (!is_file($file)) { echo "Usage: php $self MyAnimeList.xml"; exit(1); } $xml = new DOMDocument; $xml->load($file); $xpath = new DOMXPath($xml); $objects = $info = []; $childToTag = [ 'series_type' => 'Type', 'series_episodes' => 'Episodes', 'my_score' => 'Score', 'my_status' => 'Status', ]; foreach ($xpath->evaluate('//anime') as $anime) { $object = []; foreach ($anime->childNodes as $child) { $ref = &$childToTag[$child->nodeName]; if ($ref) { $object[$ref] = $child->textContent; } elseif ($child->nodeName === 'series_title') { $info[count($objects)]['title'] = $child->textContent; } elseif ($child->nodeName === 'series_animedb_id') { $info[count($objects)]['id'] = $child->textContent; } } $objects[] = $object; } $json = ['Objects' => [], 'Tags' => [], 'ObjectTags' => []]; $temps = []; foreach ($objects as $id => $object) { $temps[] = $temp = tempnam(sys_get_temp_dir(), '').'.url'; file_put_contents($temp, "[InternetShortcut]\nURL=https://myanimelist.net/anime/".$info[$id]['id']); $json['Objects'][] = [ 'CreationTime' => time(), 'FilePath' => $temp, 'Random' => mt_rand(), 'RowId' => $id + 1, 'Title' => $info[$id]['title'], ]; foreach ($object as $tag => $value) { $tag = findCreateTag($json['Tags'], [$tag, $value]); $json['ObjectTags'][] = [ 'ObjectRowId' => $id + 1, 'TagRowId' => $tag['RowId'], 'Weight' => $tag === 'Score' ? $value : 0, ]; } } echo 'Stagsi JSON 1 '.json_encode($json, /*JSON_PRETTY_PRINT+*/ 0); function findTag(array $tags, $title, $parent = null) { foreach ($tags as $ttag) { if ($ttag['ParentRowId'] === $parent and mb_strtolower($ttag['Title']) === mb_strtolower($title)) { return $ttag; } } } function findCreateTag(array &$tags, array $path) { static $nextID = 1; $parent = null; foreach ($path as $ptag) { $ttag = findTag($tags, $ptag, $parent); if (!$ttag) { $tags[] = $ttag = [ 'CreationTime' => time(), 'ParentRowId' => $parent, 'RowId' => $nextID++, 'Title' => (string) $ptag, ]; } $parent = $ttag['RowId']; } return $ttag; }

Merge and Split Databases

Two obvious operations for Copy/Paste between Stagsi processes are to merge and split databases.

Let’s imagine you have a big database and you want to extract a portion of it, for example, for sharing with someone else:

Now let’s imaging you have two databases, perhaps created at different times, that you want to merge into one (it’s generally easier to manage one database than multiple smaller ones):

Confirming Changes (Diff)

Let’s imagine you are performing a large clean-up of your database, adding and removing tags from hundreds of objects. In the end of the day, you want to confirm what exactly was changed before you commit the version.

The Copy command can be used to quickly convert part of a database into a textual representation. And for text, there are tools called “diff”/"merge" viewers: they take 2 or more files, compare them line-by-line and display the differences. For Windows, WinMerge can be used, or a similar tool from virtually any version-control system (GUIs for git, SVN, etc.). Linux has a built-in console command diff, or a GUI called meld.

Multiple Clipboards

The text inserted into the clipboard by the Copy command is self-contained – the original Stagsi process is not required for Paste into another Stagsi process to work.

This means you can paste results of the Copy command into a text editor like Notepad leaving its window open, perform another Copy followed by Paste into a new window, etc. and then go back to any of those windows, Copy everything and Paste into Stagsi later.

In an extreme case, you can even save the copied data to a text file and reuse it much later, even after a reboot or weeks of regular usage. It will work as long as file paths (to data files, thumbnail files, etc.) remain valid.

AutoHotKey

AutoHotKey, or AHK for short, is a powerful automation utility based on scripts. Here’s a script that displays a tooltip with the number of copied objects imported in linked mode:

Misc/Copy tooltip.ahk • Skip this example (#exNF) • Browse on GitHub

#Persistent OnClipboardChange: StringLeft, Start, Clipboard, 7 if (Start = "Stagsi ") { StrReplace(Clipboard, """FileLink"": null,", , CopiedCount) StrReplace(Clipboard, """FileLink"": ", , TotalCount) LinkedCount := TotalCount - CopiedCount ToolTip %LinkedCount% out of %TotalCount% objects are linked Sleep 5000 ToolTip } return

Node.js

Even though JSON is native to most scripting languages, many of them – like PHP and Node.js (JavaScript) – don’t make it easy to access the system clipboard (mostly because there’s no cross-platform way of doing so).

The simplest way is to use an external tool. In the following script we’re using nircmd (which should be in your %PATH%) to obtain Stagsi clipboard text and output the total size of all copied object files:

Misc/Copied file size.js • Skip this example (#exRA) • Browse on GitHub

var file = require('fs').mkdtempSync(os.tmpdir() + '/') + '/clip.txt' require('child_process').execSync('nircmdc64 clipboard writefile "' + file + '"') var clipboard = require('fs').readFileSync(file, {encoding: 'utf8'}) var [, data] = clipboard.match(/^Stagsi JSON \d+ (\{.+)$/s) var sum = JSON.parse(data).Objects.reduce((cur, obj) => cur + obj.FileSize, 0) console.log(`Total size of all objects: ${sum} bytes`)

Notepad 2e

Notepad 2e is a bare-bone text editor with advanced data manipulation capabilities aimed at quick execution of common one-time operations (in contrast to tools like sed and vim which shine on repetitive tasks).

For example, if you want to prepend “Game” to titles of all objects before they’re pasted:

Another example, to obtain a list of objects that came (OriginalPath) from the E:\Important directory or its subdirectories:

Spreadsheet

Suppose you want to make a neat table of your objects, perhaps for sorting, filtering or some other manipulations that spreadsheet programs are great at.

In this example we’re going to combine Notepad 2e magic with a lightweight alternative to MS Excel (1 EXE file, less than 2 MiB) called Spread32.

First, we need to format Stagsi data so that it becomes 1 line per object, with object properties separated with a Tab character:

Once you got the prepared data, turn to the spreadsheet (we’ll be assuming you want to obtain IDs for objects which file size is above or below some threshold):

Import

Duplicates & Hashes

We often refer to “duplicate” files but what does this mean exactly in Stagsi?

To answer this question, we need to briefly explain what a “hash” is (related terms are “cryptographic hash”, “digest” and “checksum”). In layman’s terms, a “hash” is short for “hash function” that, when fed some data, produces some seemingly random string of a fixed length. The string depends on the data, so that if exactly the same data is fed again, exactly the same string is produced; if the input data has changed even for a little bit, then the string will be different too.

One widely used hash function algorithm is MD5 – short for “Message Digest-5”. Contrary to the name, “message” doesn’t imply some human-readable text – it works as good for binary data (images, etc.). Here are MD5 hashes of several different strings – as you see, even a mere capitalization resulted in a drastically different hash:

string "hello": 5d41402abc4b2a76b9719d911017c592 string "Hello": 8b1a9953c4611296a827abf8c47804d7 string "HELLO": eb61eead90e3b899c6bcbe27ac581660

When you import a file, Stagsi calculates its MD5 hash and checks if the database already has an entry with the same hash. If it does, the file is considered “duplicate” and usually not imported (depending on the settings).

One obvious advantage of this approach is that it does not depend on file’s name, path, modification time or any other properties except the file’s contents. If you have 10 Terabytes worth of backups and you want to extract only non-duplicate files, it would take ages to sort by hand, especially if paths and names cannot be trusted. With Stagsi, you simply dump the entire archive and let it chew through. In the end, only files with unique contents will remain, and with some scripting you can export such objects preserving the original folder structure (using OriginalPath field and/or Import’s Regexp filters).

Note: of course, one cannot fit all possible variations of data within anyhow short string. There is a chance – called “hash collision” – that two different data would produce identical hashes; however, it is so slim for practical purposes that Stagsi ignores it completely, along with the absolute majority of other software.

Zero-Sized Files and the Null Hash

An MD5 hash of an empty “message” (input) is always d41d8cd98f00b204e9800998ecf8427e. However, Stagsi treats all empty files (0 size) as unique, assuming their Hash to be null – a special value that does not equal anything, even another null.

This allows you to have any number of empty files in your database. Such files are still separate – each has its own tags, thumbnail, RowId, etc. but when you import another empty file, it is not marked as duplicate. Use empty files where the data (file) itself has no value, only its “meta-data” (tags, attachments, etc.) is useful.

Note: remember that in SQL to compare something with NULL you should use IS, not = – SqlNULL.

Fuzzy Duplicate Images Detection

Two image files may have different hashes even though their “contents” appears to be identical. It’s a big problem when using software that puts tags into the files themselves (ahem, Adobe Bridge, ahem, Apple iTunes) because once changed, you cannot tell if two files are the same by merely comparing their contents (what hashing does) and you have to use heavy machinery for this task.

One way to compare two images is using ImageMagick’s compare utility: compare.exe -verbose -metric mae Image1.png Image2.png diff.png

You can write a script for checking a batch of files, telling you which pictures are the same dispite differences in their raw file data.

Explore Skipped

If an import session didn’t process all files, Stagsi keeps the import dialog open with the list of files reflecting the skipped items. However, it’s not very convenient to work with.

If you want to save such skipped items for later, browse them in Windows Explorer or process in some way, use the Copy command (Ctrl+C). As a result, the clipboard receives two types of data:

If you want to explore skipped files but you don’t want to copy them entirely, you can create shortcuts using nircmd:

Tags by Wildcard

Often files that we are importing were already organized into some sensible directory structure. For example, photos from this year’s birthday might be inside a folder named Birthday-2019. If there are hundreds of files, it’s nearly impossible (and unnecessary!) to tag them all manually with the same tag.

Stagsi can add tags during import based on a wildcard match. Using our photos example, we can create the following tag hierarchy:

+ Personal + Birthdays + 2018 + 2019 + Trips + 2019 + Japan + Spain

Next, we add a wildcard tagging rule for each set of tags belonging to a particular path. A “wildcard” is a simple string with two special symbols: * matching 0 or more characters (any) and ? matching exactly 1 character (any).

For example:

Remember: if you want to apply such rules after importing, you can simply re-import the same files (even if they’re in another location, they’re identified by content hashes) as long as duplicate mode is set to “append”.

Tags by Regular Expression (RegExp)

If you are importing files which already have a good directory structure, you may want to preserve some of that structure in form of tags. You can do this with wildcards, but only if the number of tags is small and/or known beforehand – because you cannot create a new tag from a file path dynamically.

This is where regular expressions come into play. They are extremely powerful but also, admittedly, more cryptic than simple wildcards. The following explanation assumes you have a basic knowledge of their syntax. Online resources can help you in building and validating expressions: uiregex.com, regex101.com, regexr.com.

Let’s imagine you are importing a bunch of game screenshots, organized by release year and game name:

...\1994\Warcraft: Orcs & Humans\Screenshot0001.png ...\1995\Command & Conquer\... ...\1995\Heroes of Might and Magic\... ...\1995\Need for Speed, The\... ...\1996\Tomb Raider\... ...\1996\Diablo\...

You want every file to be tagged with:

As you notice, out of these tags we only know 2 tags in advance: root Years and Titles tags. Others need to be extracted from the file names of each file being imported.

We can solve this with a single regular expression:

\\(\d{4})\\([^\\]+)\\

Let’s break it down:

\\start matching on any backslash (backslash is special and so needs to be doubled) (start a capture group \dmatch any digit {4}match exactly 4 instances of the preceding term )end the capture group \\match a backslash (start another capture group [^\\]match anything ([ ]) that is not (^) a backslash (\\) +match 1 or more instances of the preceding term )end the capture group \\match a backslash

The first file name on our list matches nicely, producing 2 capture groups:

...\1994\Warcraft: Orcs & Humans\... group 1 = 1994 group 2 = Warcraft: Orcs & Humans

Now we have groups that hold our desired tag names. How do we assign them?

Each regular expression tagging rule has both an expression and a replacement strings. The expression we have just made. The replacement is simpler – it’s a pipe-separated list of tag names (in long form, with parents) where $n (like $1) are replaced by the n’th group’s contents. Named groups can be also used: matched with (?<named>...) and replaced by ${name}.

So the second part of the spell is this replacement string:

Years/$1 | Titles/$2 | Years/$1/$2

…which for our first file name evaluates to:

Years/1994 | Titles/Warcraft: Orcs & Humans | Years/1994/Warcraft: Orcs & Humans

Stagsi breaks it down into 3 tags, creates tags (parents and the tag itself) as needed and assigns the tags to the file being imported.

Tags From Custom Sources

Naturally, it’s impossible for a single program to account for every existing meta-data format and consider all possible users’ preferences.

Stagsi does not try to do that. Instead, it provides core support for only a few well-known and popular meta-data formats (such as EXIF used in images) and allows you to supply tags for other situations during import using a simple JSON file containing an array of objects with keys: File (imported file’s path, relative to the JSON file), Tags (array of objects). Tags objects have two keys: mandatory Tag (array of strings – tag path) and optional Weight (defaults to 0). This example assigns two tags: Artists/"Yuno Nagasaki" (with -1 weight) and Year/1990s/1999 (with 0 weight): [ { "File": "..\\music.mp3", "Tags": [ {"Tag": ["Artists", "Yuno Nagasaki"], "Weight": -1}, {"Tag": ["Year", "1990s", "1999"]} ] } ]

If duplicate File’s exist within the same JSON or multiple JSONs, their Tags are merged as if they were specified in the same object.

The script below parses ID3v2 tags (found in audio and video files, notably used by iTunes) extracting tags from artist, album, grouping, genre, BPM and comment fields (all comma-separated) into a JSON file. Once it’s ran, you perform the usual import making sure to select the JSON file that the script has produced as a “custom meta-data source”. If your files are already in Stagsi, you can import them again in Append Tags mode to merge existing database tags with the extracted ones.

Note: even though the script supports all ID3v2 formats (2.x, 3.x, 4.x), the format itself is very old and programs often use it in non-standard ways so it may fail to extract meta-data from some files, particularly from the ancient ones.

Import/ID3v2.php • Skip this example (#exYS) • Browse on GitHub • Adjustments (2)

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $meta = processDir('C:/music'); file_put_contents('c:/tags.json', json_encode($meta)); function nameInfo(array $frames) { $res = []; $idToTag = ['TCOM' => 'Artist', 'TALB' => 'Album', 'GRP1' => 'Grouping', 'COMM' => 'Comment']; foreach ($idToTag as $id => $tag) { if (!empty($frames[$id])) { foreach ($frames[$id] as $s) { foreach (preg_split('/\s*,\s*/u', $s) as $s) { $res[] = ['Tag' => [$tag, $s]]; // This script assigns all tags the default weight of 0. // You can assign custom weight like so: //$res[] = ['Tag' => [$tag, $s], 'Weight' => 20]; } } } } if (!empty($frames['TCON'])) { foreach ($frames['TCON'] as $s) { $res[] = ['Tag' => ['Genre', $s]]; } } if (!empty($frames['TBPM'])) { foreach ($frames['TBPM'] as $s) { $res[] = ['Tag' => ['BPM', $s]]; } } return $res; } function processDir($path) { $res = []; $path = rtrim($path, '\\/').'/'; foreach (scandir($path) as $file) { $full = "$path/$file"; if (is_file($full)) { try { $frames = parseFileID3v2($full); } catch (NoID3Tag $e) { echo "no ID3v2 tag, skipping", PHP_EOL; continue; } catch (\Throwable $e) { echo '!! ', $e->getMessage(), PHP_EOL; continue; } $tags = nameInfo($frames); $tags and $res[] = ['File' => $full, 'Tags' => $tags]; } elseif (is_dir($full) and $file[0] !== '.') { $res = array_merge($res, processDir($full)); } } return $res; } // https://id3.org/id3v2.4.0-structure // https://id3.org/id3v2.4.0-frames function parseFileID3v2($file) { echo "== ", preg_replace('![/\\\\]+!u', '\\', $file), " ==", PHP_EOL; $f = fopen($file, 'rb'); $header = unpack('a3magic/nversion/Cflags/Nlength', fread($f, 3+2+1+4)); if ($header['magic'] !== 'ID3') { throw new NoID3Tag("No ID3v2 tag present."); } elseif ($header['version'] > 4<<8) { throw new Exception("Invalid ID3v2 tag version ($header[version])."); } assertNoFlags($header['flags'], 'tag'); $header['length'] = unsynchInt( $header['length'] ); echo "ID3v2 tag length: $header[length] bytes", PHP_EOL; $framesBuf = fread($f, $header['length']); fclose($f); if ($header['version'] < 3<<8) { $frames = splitID3v22Frames($framesBuf); } else { $frames = splitID3v2Frames($framesBuf); } echo "Found ", count($frames), " frames: ", join(' ', array_map(function ($frame) { return "$frame[id]-".strlen($frame['data']); }, $frames)), PHP_EOL; echo PHP_EOL; $usefulFrames = []; foreach ($frames as $frame) { // Frame types found in iTunes-tagged MP3s: // // -- 2.x -- // COM - Comments; iTunes GUI: "comments" // TAL - Album/Movie/Show title; iTunes GUI: "album" // TBP - BPM (Beats Per Minute); iTunes GUI: "bpm" // TCO - Content type; iTunes GUI: "genre" // TP1 - Lead artist(s)/Lead performer(s)/Soloist(s)/Performing group; iTunes GUI: "artist" // TP2 - Band/Orchestra/Accompaniment // TS2 - ??? // TSA - ??? // TT2 - Title/Songname/Content description; iTunes GUI: "song" // TYE - Year; iTunes GUI: "year" // // -- 3.x/4.x -- // APIC - Attached picture // COMM - Comments; iTunes GUI: "comments" // GRP1 - ???; iTunes GUI: "grouping" // MCDI - Music CD identifier // TALB - Album/Movie/Show title; iTunes GUI: "album" // TBPM - BPM (beats per minute); iTunes GUI: "bpm" // TCOM - Composer; iTunes GUI: "artist" // TCON - Content type; iTunes GUI: "genre" // TCOP - Copyright message // TENC - Encoded by // TIT2 - Title/songname/content description // TLEN - Length // TMED - Media type // TPE1 - Lead performer(s)/Soloist(s) // TPE2 - Band/orchestra/accompaniment // TPOS - Part of a set // TRCK - Track number/Position in set // TSSE - Software/Hardware and settings used for encoding // TXXX - User defined text information frame // TYER - ??? // WXXX - User defined URL link frame $id2to34 = ['COM' => 'COMM', 'TAL' => 'TALB', 'TBP' => 'TBPM', 'TCO' => 'TCON', 'TP1' => 'TCOM']; switch ($id = $frame['id']) { // 2.x default: if (!isset($id2to34[$id])) { break; } $id = $id2to34[$id]; // 3.x/4.x case 'COMM': case 'GRP1': case 'TALB': case 'TBPM': case 'TCOM': case 'TCON': //echo "> $frame[id]: ", bin2hex($frame['data']), PHP_EOL; break; echo "> $frame[id] ($id): "; assertNoFlags($frame['flags'], 'frame'); echo join('|', $strings = decodeStrings($frame['data'])); echo PHP_EOL; foreach ($strings as &$ref) { $ref = preg_replace('/^\s*|\s*$/u', '', $ref); } $usefulFrames[$id] = $strings; break; } } return $usefulFrames; } // 2.x. // https://id3.org/id3v2-00 function splitID3v22Frames($buf) { $frames = []; while (strlen($buf)) { if ($buf[0] === "\0") { break; } // padding. $header = unpack('a3id/C3length', substr($buf, 0, $hl = 3+3)); $len = ($header['length1'] << 16) + ($header['length2'] << 8) + $header['length3']; //$len = unsynchInt($len); if (strlen($buf) < $hl + $len) { throw new Exception("Not enough frame buffer."); } $frames[] = [ 'id' => $header['id'], 'flags' => null, 'data' => substr($buf, $hl, $len), ]; $buf = substr($buf, $hl + $len); } return $frames; } // 3.x and 4.x. function splitID3v2Frames($buf) { $frames = []; while (strlen($buf)) { if ($buf[0] === "\0") { break; } // padding. $header = unpack('a4id/Nlength/nflags', substr($buf, 0, $hl = 4+4+2)); // Contrary to the spec, frame size doesn't seem to be synchsafe integer, // at least in tags produced by iTunes. //$len = unsynchInt($header['length']); $len = $header['length']; if (strlen($buf) < $hl + $len) { throw new Exception("Not enough frame buffer."); } $frames[] = [ 'id' => $header['id'], 'flags' => $header['flags'], 'data' => substr($buf, $hl, $len), ]; $buf = substr($buf, $hl + $len); } return $frames; } function unsynchInt($n) { $octets = str_split(str_pad(decbin($n), 32, 0, STR_PAD_LEFT), 8); foreach ($octets as &$octet) { if ($octet[0]) { throw new Exception("Synchsafe integer ($n) expected to have no 8th bits set ($octet)."); } $octet = substr($octet, 1); } return bindec(join($octets)); } function decodeStrings($s) { $res = []; while (strlen($s)) { list($string, $tail) = decodeString($s); $res[] = $string; $s = strlen($tail) ? $s[0].$tail : ''; } return $res; } function decodeString($s) { switch ($s[0]) { case "\0": case "\3": $tail = strpos($s, "\0", 1); $tail === false and $tail = strlen($s); $string = substr($s, 1, $tail - 1); $s[0] === "\0" and $string = utf8_encode($string); return [$string, substr($s, $tail + 1)]; case "\1": case "\2": // It seems NULL byte(s) (for all string encoding types) are optional and only used if the data does // contain several strings, as a separator. For example, found in COMM only. // However, instead of using explode() we still parse strings one by one // because there must be some cases when NULLs are used as terminator, // and then this function would be useful. $tail = strlen($s); for ($i = 1; isset($s[$i + 1]); $i += 2) { if ($s[$i] === "\0" and $s[$i + 1] === "\0") { $tail = $i; break; } } $enc = $s[0] === "\1" ? 'UTF-16' : 'UTF-16BE'; return [iconv($enc, 'utf-8', substr($s, 1, $tail - 1)), substr($s, $tail + 2)]; default: throw new Exception("Unknown string encoding scheme ($s[0])."); } } function assertNoFlags($n, $type) { if ($n) { throw new Exception("ID3v2 $type flags not supported ($n)."); } } class NoID3Tag extends Exception { }

Multiple File Name Tags

Let’s suppose you have a large collection of files where their names (or parts thereof) are the actual tags. For example: sunny sunshine girl landscape public_domain.jpg. It’s not quite convenient to “parse” such a file name into individual tags using the regular technique (see the above section) because the number of tags is unknown, and you’ll need to create 1 rule per every possible tag with slightly different regexps.

Note that specifically for these reasons regexp rules allow changing the tag name delimiter, which defaults to | (because it’s not a valid symbol in file names). To extract tags from our earlier example:

As a result, first \1 is replaced with sunny sunshine girl landscape public_domain and then the delimiter is applied, making that string a list of 5 tags.

Repairing Corrupted

Sometimes database object files may go bad – power failure, linked files moved to another location, etc. Stagsi integrity checks detect this situation and mark such objects with the special Corrupted system tag.

If you have proper files somewhere, you can easily repair corrupted objects by importing them. When Stagsi detects a file (object) already contained in the database, and that object is corrupted – it replaces most of its properties (including link files and thumbnails) as if the file was imported anew, keeping existing tag assignments.

Change Data Mode

Sometimes you might want to change object import mode after the actual import. For example, files were wrongly imported as links while you wanted them to be stored, or vice versa.

If you want to convert all link-mode objects into “copy”, i.e. make a database fully independent of external files, then just run the /unlink command-line command.

For other cases, or when you want to selectively “unlink” only specific objects, you can make them “Corrupted” and then re-import with the desired mode (corrupted duplicate files are imported almost as if they were not yet known, just their tags are preserved). One simple way to “corrupt” them is removing their data files from the numbered folders (e.g. Database\1\1234.jpg or 1234 for a link) and running /check.

Surprising Uses

Audio & Video Bookmarks

By utilizing custom file formats you can realize Stagsi in many ways it wasn’t meant for in the core – for example, to tag parts of audio and video files. For this one can use M3U playlist files with extensions specific to VLC (a popular media player also called VideoLAN).

M3U is a regular text file consisting of blocks, each block addressing a specific playlist entry, which most often is just a file path, but can be preceded by comment lines (starting with #) which are often used to add custom information like playback position. Example of such a block with VLC-specific playback position (120 seconds, 2 minutes sharp):

#EXTVLCOPT:start-time=120 C:\Users\Elder\Desktop\Indie Game - The Movie.mp4

You can create such playlists by hand, via VLC’s interface (Save Playlist command, Ctrl+Y) or with the following VLC extension written in Lua which saves a playlist file and a screenshot every time you pause a video (e.g. with Space) so that in the end you can import all M3U files and add screenshots as their custom thumbnails. When you Open (Enter) an M3U file from Stagsi, it launches VLC playback exactly at that position.

Import/VLC video bookmarks.lua • Skip this example (#exQL) • Browse on GitHub • Adjustments (5)

-- To enable this extension, copy it to VLC's program folder, for example: -- C:\Program Files (x86)\VideoLAN\lua\extensions\ -- ...then (every time VLC starts) enable via the View menu. -- -- Warning: io.open() doesn't work with UTF-8 but with system-specific locale. -- Yet VLC returns playing file path as UTF-8 and have no means to convert them -- to CP1251 (or other) but even if it had the conversion would not be correct -- for all strings. Therefore Paths with non-Latin symbols will be mangled on -- save and all modes will behave as OVERWRITE (this includes video file names -- and ..._path options). local MY_NAME = 'Stagsi Playlist Maker' local NONE = 0 local ADD_NEW = 1 local APPEND = 2 local OVERWRITE = 3 -- After changing options either restart VLC or go to Tools > -- Plugins and extensions, Active Extensions tab and hit Reload extensions, -- then re-enable it via the View menu. local options = { -- Leave empty to use the video's directory. In any case, the file is prefixed -- by the video's base name (without extension) plus a unique numerical suffix. playlist_path = '', -- Warning: make sure to double backslashes! --playlist_path = 'C:\\Stuff\\Save_here', -- Like above but for video screenshots. snapshot_path = '', -- NONE doesn't write playlist files. -- -- ADD_NEW creates new M3U file. -- -- APPEND adds new lines to the first (existing) M3U file (if no file exists, -- it's created as with ADD_NEW). This allows multiple bookmarks per one -- playlist, which can be viewed and navigated in VLC's View > Playlist -- (Ctrl+L) menu. -- -- OVERWRITE wipes the file if it exists, replacing by the current (last) -- position. playlist_mode = ADD_NEW, -- Similar as above but not APPEND (works like ADD_NEW). snapshot_mode = OVERWRITE, -- Allowed VLC values - png, jpg, tiff. snapshot_format = 'png', } function descriptor() return { title = MY_NAME, version = '1.0', license = 'CC0', author = 'Soletude', description = [[ Whenever a video is paused (often with Space hotkey), saves an M3U playlist (with current playback position) and/or current video snapshot for later import into Stagsi. Open this script in a text editor to configure. ]], url = 'https://stagsi.com', capabilities = {'playing-listener', 'input-listener'}, } end -- Without these two VLC won't recognize the extension. function activate() end function deactivate() end -- From lua\modules\common.lua. local function snapshot() local vout = vlc.object.vout() if vout then -- Saves image to the location and in format specified in VLC preferences. vlc.var.set(vout, 'video-snapshot', nil) end end -- C:\foo\bar.txt -> C:\foo\ local function split_path(path) return path:gsub('[^\\/]+$', ''), path:match('[^\\/]+$') end -- path - without format (extension). A prefix will be added by vlc. -- format - without '.'. local function snapshot_to(path, format) local old = { path = vlc.config.get('snapshot-path'), prefix = vlc.config.get('snapshot-prefix'), format = vlc.config.get('snapshot-format'), seq = vlc.config.get('snapshot-sequential'), } local dir, file = split_path(path) -- VLC fails if a path has trailing \, but it also fails if it's a drive's -- root without \. So either C:\... or C:\ but not C:\...\ or C:. dir = dir:gsub('[\\/]*$', '') if not dir:find('[\\/]') then dir = dir..'\\' end vlc.config.set('snapshot-path', dir) vlc.config.set('snapshot-prefix', file) vlc.config.set('snapshot-format', format) -- vlcsnap-00001.png. vlc.config.set('snapshot-sequential', true) pcall(snapshot) vlc.config.set('snapshot-path', old.path or '') vlc.config.set('snapshot-prefix', old.prefix or '') vlc.config.set('snapshot-format', old.format or '') vlc.config.set('snapshot-sequential', old.seq or false) end local function last_file(base, suffix) local function exists(file) local f = io.open(file) return f and (f:close() or true) end -- VLC's snapshot suffixes are incremental and don't reset to 1 for new -- videos. So we see what's the last suffix that exists. for i = 1000, 1, -1 do local file = ('%s%05d%s'):format(base, i, suffix) vlc.msg.dbg(('%s: last_file(%s)'):format(MY_NAME, file)) if exists(file) then return i end end end local function handle_hotkey() vlc.msg.info(('%s: handling hotkey'):format(MY_NAME)) -- file:///C:/%20... local video_uri = vlc.input.item():uri() local video_file = vlc.strings.decode_uri(video_uri) :gsub('^[^/]*/*', '') :gsub('/', '\\') local video_dir, video_file = split_path(video_file) local video_base = video_file:gsub('%.[^.]+$', '') vlc.msg.dbg(('%s: video dir %s, file %s, base %s'):format(MY_NAME, video_dir, video_file, video_base)) if options.playlist_mode ~= NONE then local base = options.playlist_path:gsub('[\\/]*$', '') base = (base == '' and video_dir or base..'\\')..video_base..'-' local suffix = last_file(base, '.m3u') vlc.msg.dbg(('%s: playlist base %s, suffix %s'):format(MY_NAME, base, suffix or '')) if options.playlist_mode == OVERWRITE then if suffix then os.remove(('%s%05d.m3u'):format(base, suffix)) end elseif options.playlist_mode == ADD_NEW and suffix then suffix = suffix + 1 end -- Convert microseconds to seconds. local play_time = vlc.var.get(vlc.object.input(), 'time') / 1000000 local file = ('%s%05d.m3u'):format(base, suffix or 1) local block = '#EXTVLCOPT:start-time=%d\n'.. '%s\n'.. '\n' block = block:format(play_time, video_uri) local f = io.open(file, options.playlist_mode == APPEND and 'a' or 'w') f:write(block) f:close() end if options.snapshot_mode ~= NONE then local base = options.snapshot_path:gsub('[\\/]*$', '') base = (base == '' and video_dir or base..'\\')..video_base..'-' local suffix = last_file(base, '.'..options.snapshot_format) vlc.msg.dbg(('%s: snapshot base %s, suffix %s'):format(MY_NAME, base, suffix or '')) if options.snapshot_mode == OVERWRITE then if suffix then os.remove(('%s%05d.%s'):format(base, suffix, options.snapshot_format)) end elseif suffix then suffix = suffix + 1 end snapshot_to(base, options.snapshot_format) end end function playing_changed() if vlc.playlist.status() == 'paused' then local ok, msg = pcall(handle_hotkey) if not ok then vlc.msg.err(('%s: error: %s'):format(MY_NAME, msg)) end end end

Web Bookmark Manager

If you’re the kind of person that keeps hundreds of bookmarks in Chrome or Firefox, it’s likely you are not happy with the way modern browsers manage them – by folders. As such, they have all the limitations of regular file systems vs. tags.

Stagsi may be your key to the kingdom. Much like regular Windows file shortcuts (.lnk), there is a format for “Internet shortcuts” (.url). In your browser’s address bar, try dragging the website’s icon and dropping it in a Windows Explorer window – a shortcut will be created, that when launched brings you to the target web page.

And since it’s a file, it can be imported into Stagsi.

Moreover, the format is text-based and can be easily generated. Here’s a PHP script that creates .url files from Chrome’s bookmark export (in HTML format) with proper file names (becoming object title in Stagsi):

Import/Chrome bookmarks.php • Skip this example (#exOV) • Browse on GitHub • Adjustments (2)

<?php $file = 'bookmarks_1_2_20.html'; $path = 'output/'; is_dir($path) or mkdir($path, 0777, true); preg_match_all('/<DT><A HREF="([^"]+)[^>]*>([^<]+)/ui', file_get_contents($file), $matches, PREG_SET_ORDER); foreach ($matches as $match) { list(, $url, $title) = $match; $title = preg_replace('/[^\pL\pN ]+/u', ' ', $title); $title = preg_replace('/^\s*|\s*$/u', '', $title); $file = rtrim($path, '\\/')."/$title.url"; file_put_contents($file, "[InternetShortcut]\r\nURL=$url"); }

Supplimenting Regular Folders

Thanks to the linked import mode, you don’t have to move all the files into Stagsi if you are happy with the traditional file & folder structure but would only like to enhance it with tags without being forced to use Stagsi alone (whose Object Browser is inherently flat, at least for now).

You can create a database in a special subfolder (say _Stagsi) in the root folder of your data, import your files in linked mode and make the links relative (see RelativeLinks). Now, when you “launch” an object from Stagsi, it gets redirected to the file 1 level above (and outside) the database – with your files being in their original folders which you can browse as usual with Windows Explorer (or maybe with Total Commander?).

Given that Stagsi is easily Portable and small, you can even copy the entire program into that subfolder and carry it along with your data. To give an idea of the final folder layout:

Photos\Family\2019\DCIM4869.jpg - main data which is being "enhanced" Photos\Stagsi.lnk - a shortcut for quickly launching Stagsi Photos\_Stagsi\Stagsi.exe - Stagsi program files Photos\_Stagsi\Database\Stagsi.sqlite - Stagsi database files

Using Windows Shortcuts

With Stagsi, a power user must always think beyond what’s obvious to open new horizons in organizing one’s materials. When “unknown formats” setting is enabled, Stagsi accepts not only images but any kind of files – including Windows Shortcuts (.lnk files). The benefits of this:

Shortcuts are easy to create en-masse in Windows Explorer: select all the files you want to be “shortcutted” and Copy them (Ctrl+C), then right-click on some temporary folder and select Paste shortcut. Or drag & drop the files while holding right mouse button (not left) to a temporary folder – on mouse button release, you will get a menu with Create shortcuts here command.

Once you have shortcuts, import them in “copy” or “move” mode instead of “linked” to store them within the database folder, avoiding the clutter in your regular directories (like Desktop or My Documents).

Another Way To Tag Folders

If you want to avoid using shortcuts, you can link directly to folders – the only problem is that Stagsi’s import dialog is not currently suited for importing folders.

You can work around this by creating a special file in each folder (e.g. FolderMarker.txt), importing those files in linked mode, closing Stagsi and doing a Find & Replace on the database folder, replacing \FolderMarker.txt with a blank string. As a result, C:\Folder\FolderMarker.txt becomes C:\Folder and “opening” it from Stagsi opens a new Windows Explorer window directly.

Using Batch and Other Script Files

With a bit of non-standard thinking, Stagsi can be turned into a “mission control”-type program, suitable for managing thousands of commands, each identified by an evolved tag hierarchy.

Let’s imagine you have 100 of PCs that you need to maintain in some way. Each PC may have a set of batch files to perform tasks such as “reboot”, “install updates”, “make screenshot”, etc. Each batch file may have at least two tags: the PC’s name (under “PCs” root tag) and task name (under “Tasks” root tag). Whenever you want to “install updates” to a particular PC, you search for “PC/Marys” or to “Tasks/Update”, or even to “PC/Marys Tasks/Update” if you have a lot of PCs and tasks. Select your batch file in the list and run it with Enter (a hotkey to the Open command).

Multiple commands can be ran the same way – with Enter, but if you want more control, then you can export them to a temporary folder (by dragging thumbnails from Stagsi to Windows Explorer or using Ctrl+S – Export command).

Remember: Stagsi has its own type associations (per database), which default to the system ones but can be change separately. This way, if outside of Stagsi your .ps1 files are associated with VS Code for faster editing, within Stagsi you can associate them with the PowerShell interpreter directly for instant run (you can still edit the file within using the Show in Explorer command, Alt+Enter).

You can even create new associations that do not exist outside of Stagsi. Say, if you’re a heavy nircmd user then you can write your scripts with .nir extension and associate it to nircmd.exe script within Stagsi, without affecting global Windows associations. If you put nircmd.exe in the database folder, then such association becomes portable and will work on any system where Stagsi can be ran.

Command-Line

Basic Stagsi command-line syntax is:

Stagsi.exe [/ro] [/multi] [/sql [log.sql]] [[data-path] [/op...]]

By default, Stagsi launches in read/write, single instance mode, with data-path read from DataPath key in Bootstrap.json in the program’s folder (which defaults to %APPDATA%\Soletude\Stagsi\Database).

Arguments after data-path are operation-specific arguments, where the first (/op) is the operation’s identifier.

Some operations (/search, /hash, /tags, /import) are “combining”, i.e. when called without /multi and the same database (data-path) is already opened in Stagsi, that process is signaled to perform that action (e.g. open the Import dialog). Non-combining operations like /rethumb only activate the running process.

Read-Only

Sometimes you want to browse the database without making any changes. The /ro switch enables read-only mode, protecting against accidental modifications. Other consequences of this mode:

/ro is enabled automatically if the database file is not writeable (tested on launch). This allows running Stagsi from CD-ROM, network shares, etc.

Multi-Instance Mode

By default, if there is a Stagsi process running on a given database (data-path), attempt to run another process on the same database will fail, bringing the already running one to the foreground. Generally, this is what you want – keep Stagsi running in background (to preserve its caches).

By passing /multi you disable the single-instance check. However, do not run multiple processes that perform modifications – Stagsi reads most data into the memory on launch and doesn’t refresh it; when two processes change data, they may create inconsistencies. Because of this, all but one process must be ran as /ro (read-only), or at least ensure that they do not perform conflicting actions (there’s no definite list of such actions though).

Internally, when a process tries to open a database (with or without /multi), it creates a mutex and a lock file (DataPath\Lock). The lock file holds the ID (PID) of the first process that has opened this database. If the mutex could not be created, the process exits unless /multi was given. If the database wasn’t open before, then /multi has no effect and the mutex and lock file are created anyway.

SQL Log

If /sql is given, Stagsi writes all executed non-SELECT SQL statements to a text file. If there is a file argument after /sql (must not begin with / or be empty "") then that file is appended to, else a unique .sql file is created directly under DataPath.

The text file contains valid SQL instructions (alike to a database dump) and can be “applied” to an older Stagsi.sqlite database using any tool – SQLite. /sql can be used as a log journal for manual review of the database changes, as a differential backup or as a “patch” for a database on another machine (e.g. in a basic master-slave setup).

Data-Path

If this argument is missing, it’s set according to DataPath key in Bootstrap.json in the program’s folder.

If it’s given, it may be one of the following (can be conveniently used with drag & drop of a file or folder in Windows Explorer onto Stagsi’s icon):

/search

This command opens a new tab with the given search query. Apostrophes ' are replaced by quotes " to help working around weird Windows command-line quoting quirks.

This command can be used to facilitate a more specialized search in a separate program and send found results back to Stagsi in the form of “ID” search query: /22 /34 /123 /201 ... (QueryId). If Stagsi is already running, normally this would activate it and search, otherwise it will be started and search triggered.

The sample repository contains a C# + WinForms program using the official SQLite bindings to provide an “extended” search dialog which filters by most Objects table fields such as file size and hash. WinForms is used because it provides advanced text controls (spinners and data pickers) out of the box, but it’s fairly easy to rewrite the sample using WPF or UWP.

If you want to close all tabs first, you can taskkill Stagsi, delete Queries.json (this is where tab state is stored) and then run the /search commands.

/nmh

Google Chrome allows its addons to communicate with programs via a “native messaging host” protocol. By default Stagsi is coming with NMH.exe which implements it, reading messages and writing responses in an infinite loop that can be broken with Ctrl+C or Ctrl+Z (EOF).

See Chrome’s documentation for details: developer.chrome.com/extensions/nativeMe….

The included NMH.exe lets you do this:

Request object keys: do (required, one of spawn or save). For spawn, also these: args (required, an array of strings, starting from /op), wait (optional, if the response should be delayed until the process exits). For save: ext (required, new file’s extension), data (base64-encoded string).

Response object keys: error (boolean or string if have detailed error info). For spawn, also these: code (integer, exit code, only if wait and no error), clipboard (textual clipboard contents, only if wait and no error; useful for /pick). For save: file (absolute file path).

Below is a simple Google Chrome add-on that runs the search in Stagsi from the omnibox with “stg” prefix (example: “stg landscape | nature” runs Stagsi.exe /search "landscape | nature"), as well as from the toolbar button, using current page’s selection.

Troubleshooting

Use the below C version to see if the NMH program is launched, that it receives expected messages and your addon fetches its responses. This rules out various manifest-related errors.

If Chrome reports that Specified native messaging host not found then the manifest file is not registered (in the Registry) or that it’s referencing a non-existing program (path). If it says Error when communicating with the native messaging host then the host exists but it can’t be ran (e.g. it’s a .bat file, not .exe or .lnk) or that its response is empty (EOF) or bad (e.g. invalid length DWORD).

C Implementation

Google Chrome is quite peculiar about the nature of the host and doesn’t seem to call any script files (.bat, .php, etc.). For debug purposes you can use this small C/C++ program to log all input to a text file and to stdout so your addon should receive exactly what it has sent:

NMH/nmh-stub.c • Skip this example (#exMA) • Browse on GitHub • Adjustments (1)

#define _CRT_SECURE_NO_WARNINGS #include <stdio.h> #include <stdlib.h> #include <windows.h> int main(void) { FILE *debug = fopen("nmh.log", "w"); fprintf(debug, "CL: %s\n", GetCommandLine()); setbuf(debug, NULL); setbuf(stdin, NULL); setbuf(stdout, NULL); while (1) { DWORD len; if (fread(&len, 1, 4, stdin) != 4) { return 1; } fprintf(debug, "in: %u\n", len); char *buf = (char*) malloc(len + 1); if (fread(buf, 1, len, stdin) != len) { return 2; } buf[len] = 0; fprintf(debug, " %s\n", buf); fwrite(&len, 1, 4, stdout); fwrite(buf, 1, len, stdout); } }

Here’s a compiled version (2 KiB): github.com/Soletude/Stagsi-Coo….

/hash

This command allows locating one or more files in a database. For example, to add a new command to Windows Explorer’s context menu that displays the selected file in Stagsi (if it’s part of the database) import this .reg file:

Misc/Find in Stagsi.reg • Skip this example (#exJN) • Browse on GitHub • Adjustments (1)

Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\SOFTWARE\Classes\*\shell\Find in Stagsi\command] @="\"C:\\Program Files (x86)\\Stagsi\\Stagsi.exe\" /hash \"%1\""

If you’d rather have it under the Send To menu then add a shortcut (.lnk) to %USERPROFILE%\SendTo folder, which on Windows 10 resolves to C:\Users\your_user_name\AppData\Roaming\Microsoft\Windows\SendTo. Using nircmd: nircmd shortcut "Stagsi.exe" ~$sys.userprofile$\SendTo "Find in Stagsi" /hash

Let’s say you’re using Stagsi as a “cataloguing” replacement for WinAMP – you use the latter to play media files but you queue and manage them using Stagsi using a Tools script like so:

Tools/Selection to &WinAMP.php • Skip this example (#exTU) • Browse on GitHub • Adjustments (1)

<?php $data = json_decode(strstr(file_get_contents(getenv('SELECTION')), '{')); $cl = ['C:/Program Files/winamp/winamp.exe']; foreach ($data->Objects as $object) { $cl[] = $object->FilePath; } system(join(' ', array_map('escapeshellarg', $cl)));

If you want to change tags (e.g. rating) of the currently playing track, use WinAMP’s Explore item(s) folder (Ctrl+F in Playlist Editor) to locate its file and then use Explorer’s Find in Stagsi command set up above to select it in Stagsi.

/import

This command displays the Import dialog with last import settings and file list read from the command-line (recursively adding files from directories).

Let’s imagine you are a NES addict and you play games in FCEUX (an open source NES emulator). FCEUX supports scripting in Lua language. Here’s how you can bind the Z key for taking screenshot and pushing it to Stagsi in one keystroke:

Import/FCEUX screenshots.lua • Skip this example (#exLJ) • Browse on GitHub • Adjustments (3)

local temp = os.getenv('TEMP')..'\\stagsi-fceux' local last = 0 while true do if input.get().Z and os.difftime(os.time(), last) >= 1 then gui.savescreenshotas(temp) emu.frameadvance() os.execute('"C:\\Program Files\\Stagsi\\Stagsi.exe" /import '..temp..'') os.remove(temp) last = os.time() else emu.frameadvance() end end

/pick

This command provides a very simple interface for using Stagsi as a kind of “open file” dialog.

With /search you can open a new tab (as in the /op of /search).

With /force you can add hidden criteria using the same query syntax (/force is ignored on error). Using this option alone is useful to limiting a Stagsi instance to a specific subset of tags.

With /count you can specify how many results you require to be returned (min defaults to 1, max – to “infinity”).

Stagsi exits with code 0 if user has picked object(s) meeting these criteria (/force and /count) – in this case clipboard is changed as if the Copy command was used allowing the caller to retrieve picked files’ paths, description as a JSON or first valid image’s data.

Below is a simple C program (can be compiled with MSVS or gcc under cygwin) that asks Stagsi for a wallpaper image and then sets it to all displays obtaining its path from Windows Explorer’s list of “pasted” files:

Misc/Pick wallpaper.c • Skip this example (#exPO) • Browse on GitHub • Adjustments (2)

#include <stdio.h> #include <windows.h> int main(int argc, char** argv) { // Attention: CreateProcessW (even if it's not used in this example) // requires lpCommandLine to be writeable. char cl[MAX_PATH]; snprintf(cl, MAX_PATH, "\"%s\" /pick /search \"%s\" /force \"%s\" /count 1 1", "C:\\Program Files\\Stagsi\\Stagsi.exe", "Wallpapers", "(PNG | JPG | GIF | BMP)"); STARTUPINFO si = {0}; PROCESS_INFORMATION pi = {0}; BOOL processOK = CreateProcess(NULL, cl, NULL, NULL, FALSE, 0, NULL, NULL, &si, &pi); WaitForSingleObject(pi.hProcess, INFINITE); DWORD code; if (!GetExitCodeProcess(pi.hProcess, &code)) { code = (DWORD) -1; } if (code == 0) { // Successfully picked an image. OpenClipboard(NULL); HANDLE drop = GetClipboardData(CF_HDROP); wchar_t fn[MAX_PATH]; if (drop != NULL && DragQueryFileW(drop, 0, fn, MAX_PATH - 1)) { BOOL wallpaperOK = SystemParametersInfoW(SPI_SETDESKWALLPAPER, 0, fn, SPIF_UPDATEINIFILE | SPIF_SENDCHANGE); if (wallpaperOK) { CloseClipboard(); return 0; } } CloseClipboard(); } return 1; }

Maintenance

There are a number of service operations that you normally don’t need to call in a default setting.

/op Meaning Affected system tag When to call
/check Calculate hashes for object data on disk and compare with ones in the database. Corrupted When you have disabled BackgroundChecks setting and/or want to manually trigger the integrity check.
/rehash Abandons object data hashes (and file sizes) from the database in favour of existing data on disk. Hashes are preserved for objects with a missing or a zero-sized data file, and for ones whose new hash is already contained in the database (i.e. another object has the same hash). Corrupted When object data in Data path (or linked target files) was changed and you are sure the new state should be perceived as valid from now on.
/rerand Randomize all database objects, effectively changing the Random field. None When you want the Order by Random sort mode to generate new order.
/rethumb Regenerates thumbnails according to the current state of the database and files. Custom Thumbnail When manual changes were made to Thumb... fields in the database or to t, u, c-prefixed files in the Data path.
/unlink Replace all objects in linked mode with non-linked mode, copying targeted file data into the database. None When you want a database to become self-contained, not relying on external data files.

System Scheduler

You can schedule running maintenance tasks with the standard Task Scheduler of Windows. It’s a powerful tool with a friendly GUI.

Below is a sample .xml task configuration (using the GUI or the command-line) that runs /dbcheck, /rerand and /check every 3 days at night (2 AM) – longer-running commands last so as not to cause others to be never ran since they have less chances of finishing.

Misc/Maintenance scheduler.xml • Skip this example (#exUB) • Browse on GitHub • Adjustments (5)

<?xml version="1.0" encoding="UTF-16"?> <Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task"> <Triggers> <CalendarTrigger> <StartBoundary>2019-01-01T02:00:00</StartBoundary> <Enabled>true</Enabled> <ScheduleByDay> <DaysInterval>3</DaysInterval> </ScheduleByDay> </CalendarTrigger> </Triggers> <Settings> <WakeToRun>true</WakeToRun> </Settings> <Actions Context="Author"> <Exec> <Command>"C:\Program Files\Stagsi\Stagsi.exe"</Command> <Arguments>/dbcheck</Arguments> </Exec> <Exec> <Command>"C:\Program Files\Stagsi\Stagsi.exe"</Command> <Arguments>/rerand</Arguments> </Exec> <Exec> <Command>"C:\Program Files\Stagsi\Stagsi.exe"</Command> <Arguments>/check</Arguments> </Exec> </Actions> </Task>

The configuration can be imported via the GUI or by this command, and it will process the database of the importing user:

schtasks /create /xml "stagsi.xml" /tn "Stagsi Maintenance"

Partial /rehash

As of now maintenance commands always process the entire database and cannot be limited to processing specific objects only. If your database is large and its files change often (maybe because you have imported WindowsLNK), you might be in need of updating hashes of selected objects only.

This script accepts the list of objects’ RowId’s (conveniently obtained using the Property panel’s RowId input’s Copy button) and updates their hashes:

Database/rehash.php • Skip this example (#exXU) • Browse on GitHub

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $self = array_shift($argv); $dbPath = array_shift($argv); $ids = $argv; if (!is_file($dbPath) or !$ids) { echo "Usage: php ", basename($self), " stagsi.sqlite id [id id ...]", PHP_EOL; exit(1); } chdir(dirname($dbPath)); $settings = json_decode(file_get_contents('Settings.json')); $db = new PDO("sqlite:$dbPath"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $getStmt = $db->prepare("SELECT * FROM Objects WHERE rowid = ?"); $setStmt = $db->prepare("UPDATE Objects SET Hash = ? WHERE rowid = ?"); foreach ($ids as $id) { $id = (int) $id; $getStmt->bindValue(1, $id); $getStmt->execute(); $row = $getStmt->fetchObject(); if (!$row) { echo "$id: no such row", PHP_EOL; continue; } $base = floor($id / $settings->FolderSize)."/$id"; $file = is_file($base) ? file_get_contents($base) : "$base.$row->Format"; if (!is_file($file)) { echo "$id: no data file: $file", PHP_EOL; continue; } $setStmt->bindValue(1, $hash = md5_file($file)); $setStmt->bindValue(2, $id); $setStmt->execute(); echo $hash === $row->Hash ? "$id: unchanged" : "$id: updated", PHP_EOL; }

The following alternative version does the same but is called from the Tools menu and updates hashes of the objects selected in Object Browser (Stagsi is closed and reopened on completion). The script’s file name must start with Selection.

Tools/Selection rehash.php • Skip this example (#exLZ) • Browse on GitHub

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $selection = json_decode(strstr(file_get_contents(getenv('selection')), '{')); system('taskkill /pid '.escapeshellarg(getenv('pid'))); for ($i = 0; $i < 50; ++$i) { $line = system('tasklist /fi "pid eq '.escapeshellarg(getenv('pid')).'" /fo csv'); if (strpos($line, '","') === false) { $i = -1; break; } usleep(100000); } if ($i >= 0) { echo "cannot terminate Stagsi", PHP_EOL; exit; } $db = new PDO("sqlite:Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $setStmt = $db->prepare("UPDATE Objects SET Hash = ? WHERE rowid = ?"); foreach ($selection->Objects as $obj) { $id = (int) $obj->RowId; if (is_file($obj->FilePath)) { $setStmt->bindValue(1, $hash = md5_file($obj->FilePath)); $setStmt->bindValue(2, $id); $setStmt->execute(); } } system('start "Stagsi" '.escapeshellarg(getenv('stagsi')));

Note: these scripts do not remove the Corrupted system tag. To do so, view “corrupted” object(s) in Stagsi and it will soon correct the state of this tag in background according to the new hash.

Other Partial Commands

We have PartialRehash a few useful scripts for performing partial /rehash. For other commands, you can either roll your own script or use this trick:

Settings

Not all Stagsi settings are accessible via the GUI. You can change everything it has via the Settings.json file using a text editor like Windows Notepad (while the program is not running).

The following settings exist:

Name Default Meaning Effect Change implications
BackgroundChecks True If enabled, recently viewed objects are added to a queue. When Stagsi is idling, it calculates hashes for objects in this queue and compares to the Hash field in the database, adding/removing the Corrupted system tag accordingly. If disabled, browsing might be more performant on very low-end hardware (even though it’s designed to be non-intrusive and working even over USB) but you won’t be notified of data corruptions unless you trigger the check manually (/check). None.
CacheImagesInMemory True Preloads thumbnails you are likely to see in advance from disk. If enabled, the disk experiences higher activity but scrolling is smoother. None.
FolderSize 1000 An approximate number of files per each folder. The real maximum is about 4 times that, if every single object has a standard, user and crop thumbnails. If made too small or too large, everything may be slowed down. Will fail to locate already imported objects and their thumbnails until they are manually moved around to respective folders (there is no built-in command for that).
FreeSpaceThreshold 100 MiB Affects import and other operations that may write a large number of data to disk. Stagsi makes a guess at the beginning of such operation and bails out if the remaining space of the disk where the database is located is less than this value. If set to 0, no such check happens but import may fail in the middle with your system running completely out of space. None.
ImportUnknownFormats True Allows importing every type of file, including files for which Stagsi cannot generate thumbnails automatically. If enabled, files of “unknown” formats still work but you’ll see placeholder images instead of actual thumbnails, unless you add one manually. None.
LargeFileThreshold 10 MiB Used in two operations: during import in Auto mode (files below this value are copied, others – linked to) and hash calculation (only last – note: last! – this many bytes of a file are hashed). Affects import in the explained way. May render already imported objects’ hashes invalid, mistakenly marking them with Corrupted. This can be fixed by running /rehash (assuming no object has a truly corrupted data).
MaxImageCacheSize 1.5 GiB An approximate maximum amount of RAM that Stagsi may use per process. Note that Stagsi is a Net program and this limit isn’t guaranteed to be very accurate. Low value may worsen the performance. Overly high value may cause your system to be unstable due to lack of memory. None.
MaxThumbWidth 1000 pixels When Stagsi generates a thumbnail (the t file prefix) and it’s above this dimensions, it’s scaled down. Directly linked to your zoom level. If your thumbnails are larger than this value, they will look blurry. But making this setting very large will cause high memory usage and poor performance. Should be set to the maximum (or slightly less) thumbnail dimension that you expect to ever use during browsing. Will be in effect for newly imported objects or after /rethumb.
NoThumbThreshold 20 MiB (200 MiB prior to 1802) Max file size for known formats for which thumbnails are generated and metadata¹ is extracted. Files larger than this will get no automatic thumbnail and tags¹. Setting it too high may cause out of memory errors during import.

¹ Since Stagsi version 1802, this only affects metadata detection. Thumbnails are now generated for files of any size.

None.
ScaleStep 1.5 The factor by which thumbnail size changes when using the buttons and hotkeys in the browser. Low value – more gradual zooming, requiring more clicks to get to a higher/lower zoom level. None.
ThumbFormat JPG One of png or jpg. Specifies the format for standard, user and cropped thumbnails. PNG is best for sharp images and pixel art but has larger file size (leading to higher memory usage and potentially lower performance). JPEG is best for photos but produces blurry images for low-resolution images like pixel art. No thumbnails will be visible for already imported objects until /rethumb is called.
ThumbQuality 80 Only if ThumbFormat is jpg. Sets image quality in % where 100 (%) is the best possible quality – marginally better than 95 or even 90 but much larger in size. Low value – low picture quality, small thumbnail files (less memory usage, better performance). Existing thumbnails will use old quality until /rethumb is called.
Language User interface language as a two-letter code (e.g. ru for Russian). Empty value uses the user’s display language. If the requested localization is unavailable – English is used. Affects most displayed texts. The cookbook may not be available in all supported languages. Some texts are used during database initialization only (e.g. system tag names) and do not change with this setting.

License Key

Stagsi may operate in two basic modes: limited (freeware) and unlimited (licensed). Effective license key is read from two locations, in this order:

These are the only locations where license info is stored. To activate a license put a key file there and restart Stagsi (if it was running); there is no online activation or other requirements. If no valid key files exist then the Free edition is made effective.

License key is a data packet signed by one of the Soletude’s RSA keys, public parts of which you can obtain from here: go.soletude.ca/stagsi/license/rsa (they are also hardcoded into Stagsi). To decode:

These keys are known:

Other Tips

Exporting to Spreadsheet

Stagsi can be used as an intermediate tagging/management utility – it takes some data, you tag it, then give results to some other tool. Luckily, open formats make it very easy both to feed and to pull the data.

Here is a single SQL query that produces a table of 1 row per object, with the information like its title, hash and list of tags (only titles, no parents):

SELECT o.Title, o.Hash, o.FileSize, o.Format, GROUP_CONCAT(t.Title, '"') FROM Objects o JOIN ObjectTags ot ON ot.ObjectRowId = o.RowId JOIN Tags t ON t.RowId = ot.TagRowId GROUP BY o.RowId

By using it with sqlite3 you can create a file that can be opened in any program understanding CSV (“comma-separated values”) format – note the triple " which is cmd.exe’s way to escape the quotes:

Export/Spreadsheet.bat • Skip this example (#exDT) • Browse on GitHub • Adjustments (1)

sqlite3.exe -csv Stagsi.sqlite "SELECT o.Title, o.Hash, o.FileSize, o.Format, GROUP_CONCAT(t.Title, '"""') FROM Objects o JOIN ObjectTags ot ON ot.ObjectRowId = o.RowId JOIN Tags t ON t.RowId = ot.TagRowId GROUP BY o.RowId"

For MS Excel you have to use ; instead of , as the separator:

Export/Excel.bat • Skip this example (#exQA) • Browse on GitHub • Adjustments (2)

sqlite3.exe -csv -separator ";" Stagsi.sqlite "SELECT o.Title, o.Hash, o.FileSize, o.Format, GROUP_CONCAT(t.Title, '"""') FROM Objects o JOIN ObjectTags ot ON ot.ObjectRowId = o.RowId JOIN Tags t ON t.RowId = ot.TagRowId GROUP BY o.RowId"

We are using quotes (") to separate tag names because quotes is the only symbol not valid in a tag title and it cannot be escaped.

Exporting with Keywords

Unlike other popular tools like Adobe Bridge and Apple iTunes, Stagsi does not store the meta-data within the file (image or other) itself. This means if you open the file outside of Stagsi (or copy it by means of the Export, Ctrl+S command) you don’t see any of your tags.

There are many reasons against modifying files in-place, one being that Stagsi’s tag hierarchy is complex and it’s unclear how to fit it into plain-text keyword fields that not even some formats have (e.g. ID3 audio tags don’t) – or that they properly support Unicode symbols (most of meta-data formats are ancient).

However, in some cases you wish to export images with a simplified tag layout and/or fill other meta-data properties. Here’s a PHP script that is using built-in iptcembed() function to add keywords ands authorship (identified by children of a specific tag) to JPEG images (only JPEG) that will show up in Windows Explorer’s file properties:

Export/With keywords.php • Skip this example (#exMS) • Browse on GitHub • Adjustments (3)

<?php $outputPath = "Desktop\\Exported-with-keywords"; $stagsiDataPath = "Database\\"; $db = new PDO("sqlite:$stagsiDataPath/Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $getTags = $db->prepare(" SELECT Title FROM ObjectTags ot JOIN Tags t ON ot.TagRowId = t.RowId WHERE ot.ObjectRowId = ? "); $stmt = $db->prepare("SELECT RowId FROM Tags WHERE Title = 'Author'"); $stmt->execute(); list($authorParentId) = $stmt->fetch(); $stmt = $db->prepare("SELECT RowId FROM Objects WHERE Format = 'jpg'"); $stmt->execute(); while ($row = $stmt->fetch()) { list($rowid) = $row; $getTags->bindValue(1, $rowid); $getTags->execute(); $base = "$stagsiDataPath/".floor($rowid / 1000)."/$rowid"; $file = is_file($base) ? file_get_contents($base) : "$base.jpg"; $output = "$outputPath/$rowid.jpg"; copy($file, $output); $iptc = ''; while ($tag = $getTags->fetch()) { list($title, $parent) = $tag; // 0x02 is record ID, 0x50 and 0x19 are dataset IDs (a byline, a keyword). $prefix = $authorParentId == $parent ? '1C0250' : '1C0219'; $iptc .= pack('H*n', $prefix, strlen($title)).$title; } file_put_contents($output, iptcembed($iptc, $output)); }

EXIF specification: www.exif.org/Exif2-2.PDF.

IPTC specification with XMP node names: iptc.org/std/IIM/4.1/specifi….

Distributed (Shared) Database

There are several ways to allow multiple people (computers) to work with the same Stagsi database. However, in all cases only 1 user must be modifying the database at the same time, otherwise changes of other users will be lost.

The Tools menu can help with the VCS approach – create a simple .bat file and place it within the Tools directory in the database folder to get a new menu item which will commit and push (publish) your changes in two clicks (if you don’t want to enter the message every time, add -m YOUR_MESSAGE after git commit):

Tools/Git add, commit and push.bat • Skip this example (#exUV) • Browse on GitHub • Adjustments (1)

git add -A git commit git push rem or: git push -m YOUR_MESSAGE

Database Compartments

Generally, a single database wins over multiple small databases: it has a unified tag tree, ability to search among everything on one page, etc. However, extremely big databases (100,000s of objects) or portable databases (or BYOD systems) will often combine files in linked mode from multiple sources (e.g. USB HDDs or network shares), some of which may not be available all the time (e.g. only at home or at work).

Stagsi doesn’t mind missing files but two problems arise:

The /pick command may be used to address both issues:

As a result, Stagsi will ensure that all results no matter the user’s search query match the /force’d one and consequentally mismatching (= unavailable) files will never be queued for integrity checks (which happens only when an object is displayed) and marked Corrupted.

Shared Per-User Database

Suppose you have a single large database and multiple users which need to work with it, each in their own context (search tabs, window positions, etc.). For example, a PC is used by several people and you want them to use the same underlying database files (like Stagsi.sqlite) while keeping their contexts (JSON files) per-user (in %APPDATA%).

Stagsi doesn’t allow splitting the database natively but NTFS symlinks and directory junctions can help you. First, move the database into a shared folder, e.g. to C:\Stagsi. Ensure Bootstrap.json points to some per-user location (e.g. to %APPDATA%\Stagsi as it does by default in non-portable versions). Then, in every per-user folder create symlinks and junctions so that:

%APPDATA%\...\Stagsi.sqlite ===points to===> C:\Stagsi\Stagsi.sqlite %APPDATA%\...\Settings.json ===points to===> C:\Stagsi\Settings.json %APPDATA%\...\0 ===points to===> C:\Stagsi\0 %APPDATA%\...\1 ===points to===> C:\Stagsi\1 ...

Files which remain not symlinked (such as Windows.json) will be created in the per-user location, not in the shared folder. When Stagsi opens such a database (not by the shared path – C:\Stags\Stagsi.sqlite – but by a per-user location) it will read linked files and folders (Stagsi.sqlite, etc.) from C:\Stagsi and read all other files from the per-user location (in %APPDATA%).

Attachments

Let’s imagine you have a collection of video lessons or audio podcasts. You want to tag them, and you import them as usual. But while watching or listening to them, you might want to keep notes or create bookmarks at different positions (see our VLC extension), and you want them to remain connected to the original file.

One way is to import your notes into Stagsi and use tags to group notes with their video files. But this is tedious:

Attachments offer a better way to do this. They are not part of the usual Stagsi operations – integrity checks, tagging, search, etc. They don’t have thumbnails and you cannot “import” them. They are regular files and folders placed into special “compartments” (subfolders) that Stagsi reserves for every object in its database.

Attachments are managed via object’s context menu in the browser. You are also free to manipulate each “compartment” folder via Windows Explorer or other program, even while Stagsi is running.

Attachments processes are started within CommonScriptEnv which is particularly useful for scripts and .bat files.

“Has Attachments” Tag

Stagsi doesn’t track attachments and doesn’t automatically maintain a special tag like “Custom Thumbnail”. As a result, you can’t search based on whether an object has attachments or not. This is because attachment directories may be (and are even encouraged to be) changed from the outside of Stagsi, e.g. using Windows Explorer.

If you need such a tag but do not want to assign it manually, use the following PHP script:

Tools/Update Has Attachments tag.php • Skip this example (#exRN) • Browse on GitHub • Adjustments (2)

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $hasAttachmentsTagRowid = 12; $stagsiDataPath = "C:/foo/stagsi_debug/db"; $pid = escapeshellarg(getenv('PID')); exec('taskkill /pid '.$pid); do { exec('tasklist /fi "pid eq '.$pid.'" | find "INFO"', $o, $c); usleep(100000); } while ($c); $db = new PDO("sqlite:$stagsiDataPath/Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $db->exec("DELETE FROM ObjectTags WHERE TagRowId = $hasAttachmentsTagRowid"); $stmt = $db->prepare("INSERT INTO ObjectTags (ObjectRowId, TagRowId) VALUES (?, $hasAttachmentsTagRowid)"); foreach (scandir("$stagsiDataPath/Attachments") as $group) { if (ltrim($group, "0..9") !== "") { continue; } foreach (scandir("$stagsiDataPath/Attachments/$group") as $id) { if (ltrim($id, "0..9") !== "") { continue; } if (count( scandir("$stagsiDataPath/Attachments/$group/$id") ) > 2) { $stmt->bindValue(1, $id); $stmt->execute(); } } }

Attached Queries

Suppose you have a bunch of videos grouped by series. For example: Black Mirror Season 1 The National Anthem Fifteen Million Merits The Entire History of You Season 2 Be Right Back White Bear The Waldo Moment White Christmas

You have many top-level tags (“Westworld”, “Doctor Who”, etc.) and each has numerous sub-tags and sub-sub-tags. You also have a special tag “Cover” which is assigned to exactly 1 item of the series (for example, first season’s first episode or the “best” episode of the series). When you’re looking for something to watch, you include this “Cover” tag into the query so that the same series does not appear multiple times in the results.

When you’ve decided on a series, you may want to “enter” into it, searching for its episodes only. This involves:

If you do this by hand, it will quickly become tedious.

First, you can create an attached .bat file for every object which is part of some logical group (e.g. same video series). It would open one or more tabs in Stagsi. Here, first line opens a tab with Black Mirror videos while the second line opens its sound track: "%STAGSI%" /search "Black Mirror" MP4 "%STAGSI%" /search "Black Mirror" MP3

Advantage: flexibility; you can put any command and have it called with a few keystrokes. Disadvantage: the same script should be attached to every object and if edited must be updated everywhere (but this can be rectified to a point with fsutil).

Second, you can use Tools if you know that all your video-related queries are constructed the same way (e.g. “Video” tag + name of the series under the “Series” root tag, for example: Video Series/"Black Mirror"). Name this script Selection &episodes.php and call it quickly with Alt+T,Alt+E:

Tools/Selection &episodes.php • Skip this example (#exQZ) • Browse on GitHub • Adjustments (2)

<?php $selection = json_decode(strstr(file_get_contents(getenv('selection')), '{')); $seriesTagRowId = null; $tags = []; foreach ($selection->Tags as $tag) { $tags[$tag->RowId] = $tag; if (!$tag->ParentRowId and $tag->Title === 'Series') { $seriesTagRowId = $tag->RowId; } } $queries = []; foreach ($selection->ObjectTags as $obj) { if ($tags[$obj->TagRowId]->ParentRowId === $seriesTagRowId) { $queries[$obj->TagRowId] = "\"{$tags[$seriesTagRowId]->Title}\"/\"{$tags[$obj->TagRowId]->Title}\""; } } $query = '('.join(' | ', $queries).') Video'; exec(escapeshellarg(getenv('stagsi')).' /search '.esc($query)); // escapeshellarg() simply drops "s. function esc($s) { return '"'.preg_replace('/"/u', '"""', $s).'"'; }

You can of course combine both approaches, using Tools for all but some objects requiring special treatment via an attached script.

Icon Variants

Let’s assume you have a database of pretty icons for GUI design. Raster icons typically use the PNG format but you may have ICOs too, and even those PNGs can be of different sizes (say 16x16 and 32x32). It makes sense to import only one particular variant into Stagsi (say 32x32 PNGs) and add others as attachments to avoid polluting search results with icons different in dimensions only.

But what if you got thousands of icons? For example, the free FatCow iconset (an excellent one, by the way – www.fatcow.com/free-icons) has almost 4000 icons, in 11 color variations, that’s not including ICOs. Attaching them all by hand would be insane.

Here’s a helper script to do that for FatCow. It will work for other cases as long as the variations are placed into a single folder (e.g. FatCow_Icons16x16 for 16x16 PNG, FatCow_Icons16x16_Color\Blue for same images but in a blue theme, etc.). Feel free to adapt it to your needs:

Database/Attachments.php • Skip this example (#exRB) • Browse on GitHub • Adjustments (5)

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); $attPath = 'C:/Stagsi DB/Attachments'; $db = new PDO("sqlite:$attPath/../Stagsi.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $getStmt = $db->prepare("SELECT RowId FROM Objects WHERE Hash = ?"); $walk = function ($path, $basename) use ($getStmt, $db, $attPath) { foreach (scandir($path) as $icon) { if (is_file("$path/$icon")) { $orig = "F:/Icons/FatCow/FatCow_Icons32x32/$icon"; $getStmt->bindValue(1, md5_file($orig)); $getStmt->execute(); $id = $getStmt->fetchObject()->RowId; $getStmt->closeCursor(); $atts = "$attPath/".floor($id / 1000)."/$id"; is_dir($atts) or mkdir($atts, 0755, true); copy("$path/$icon", "$atts/$basename.png"); } } }; // This script assumes that Stagsi database has 32x32 PNGs imported and you // want to add (attach) 16x16, gray 16x16 and gray 32x32 as variants. $walk("F:/Icons/FatCow/FatCow_Icons16x16", '16x16'); $walk("F:/Icons/FatCow/FatCow_Icons16x16_Grey", 'Disabled 16x16'); $walk("F:/Icons/FatCow/FatCow_Icons32x32_Grey", 'Disabled 32x32');

Web Database Viewer

Below is a PHP database browser. It’s extremely simplistic but can serve as a basis for a full-scale implementation (but note that Stagsi already includes an easy sharing function with a similar online browser).

Pertinent features and cautions about the below code:

With this, the code:

Misc/Web viewer.php • Skip this example (#exGJ) • Browse on GitHub • Adjustments (3)

<?php $stagsiDataPath = 'C:/Stagsi Demo/db'; function nodeToWHERE(Stags\Query\Node $node, PDO $db) { if ($node instanceof Stags\Query\ListNode) { $res = []; foreach ($node->children as $child) { $res[] = '('.nodeToWHERE($child, $db).')'; } return join($node->isAnd ? ' AND ' : ' OR ', $res); } else if ($node instanceof Stags\Query\IdNode) { return 'o.RowId '.($node->isNegative ? '<>' : '=').' '.((int) $node->id); } else if ($node instanceof Stags\Query\TagNode) { if ($node->isRecursive) { throw new Exception("Recursive (~) tag search is not supported."); } return 'LOWER(t.Title) '. ($node->isNegative ? '<>' : '=').' '. strtolower($db->quote(end($node->path))); } else { throw new Exception("Unknown query node type."); } } set_error_handler(function ($severity, $msg, $file, $line) { throw new \ErrorException($msg, 0, $severity, $file, $line); }, -1); extract($_REQUEST + [ 'query' => '', 'sort' => 'rowid', 'desc' => true, 'view' => 'thumbs', 'thumb' => '', 'page' => 1, ], EXTR_PREFIX_ALL, 'r'); if ($r_thumb = (int) $r_thumb) { chdir($stagsiDataPath); $path = floor($r_thumb / 1000)."/[cut]$r_thumb.*"; $files = glob($path); $files = array_combine(array_map(function ($f) { return basename($f)[0]; }, $files), $files); $files += ['c' => '', 'u' => '', 't' => '']; $file = $files['c'] ?: ($files['u'] ?: $files['t']); $info = getimagesize($file); header("Content-Type: $info[mime]"); readfile($file); exit; } $db = new PDO("sqlite:$stagsiDataPath/db.sqlite"); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // Obtain from https://github.com/Soletude/Stags-Query-Parser require_once 'Stags-Query-Parser/Stags.Query.php'; $where = nodeToWHERE((new Stags\Query\Parser($r_query))->parse(), $db) ?: '1'; ltrim($r_sort, 'a..z') === '' or $r_sort = 'rowid'; $sort = "o.$r_sort"; $desc = $r_desc ? 'DESC' : 'ASC'; $sql = <<<SQL SELECT o.RowId, o.Title, GROUP_CONCAT(t.Title, '"') TagTitles FROM Object o JOIN dataobjecttags ot ON ot.dataobject_rowid = o.RowId JOIN Tags t ON t.RowId = ot.tag_rowid WHERE $where GROUP BY o.RowId ORDER BY $sort $desc LIMIT :start, :count SQL; $stmt = $db->prepare($sql); $stmt->bindValue('start', ($r_page - 1) * 100); $stmt->bindValue('count', 100); $stmt->execute(); $qs = '?'.$_SERVER['QUERY_STRING'].'&'; ?> <!DOCTYPE html> <html> <head> <title>Simple Stagsi Web Viewer</title> <style> .group { margin-right: 1em; } .results.thumbs img { height: 100px; } .results.list { text-align: center; } </style> </head> <body> <h1><?=htmlspecialchars($stagsiDataPath)?></h1> <form action=""> <label class="group"> <b>Search query:</b> <input value="<?=htmlspecialchars($r_query)?>" name="query"> </label> <span class="group"> <b>Sort by:</b> <select name="sort"> <option value="rowid" <?=$r_sort == 'rowid' ? 'selected' : ''?>>Object ID</option> <option value="random" <?=$r_sort == 'random' ? 'selected' : ''?>>Random</option> <option value="title" <?=$r_sort == 'title' ? 'selected' : ''?>>Title</option> <option value="filesize" <?=$r_sort == 'filesize' ? 'selected' : ''?>>File size</option> <option value="format" <?=$r_sort == 'format' ? 'selected' : ''?>>Format</option> </select> <label> <input type="hidden" name="desc" value="0"> <input type="checkbox" name="desc" value="1" <?=$r_desc ? 'checked' : ''?>> Descending </label> </span> <span class="group"> <b>View as:</b> <label> <input type="radio" name="view" value="thumbs" <?=$r_view === 'thumbs' ? 'checked' : ''?>> Thumbnails </label> <label> <input type="radio" name="view" value="list" <?=$r_view === 'list' ? 'checked' : ''?>> List </label> </span> <button type="submit">Apply</button> </form> <p> <a href="<?=htmlspecialchars("$qs&page=".($r_page + 1))?>">Next page</a> | <?php if ($r_page > 1) {?> <a href="<?=htmlspecialchars("$qs&page=".($r_page - 1))?>">Previous page</a> <?php }?> | <a href="?">Reset all</a> </p> <div class="results <?=htmlspecialchars($r_view)?>"> <?php while ($row = $stmt->fetchObject()) {?> <?php if ($r_view === 'list') {?> <hr> <a href="<?=htmlspecialchars("?thumb=$row->RowId")?>"> <img src="<?=htmlspecialchars("?thumb=$row->RowId")?>"> </a> <p><?=htmlspecialchars($row->Title)?></p> <p> <?php foreach (explode('"', $row->TagTitles) as $tag) {?> [ <a href="<?=htmlspecialchars("$qs&query=\"$tag\"")?>"> <?=htmlspecialchars($tag)?></a> ] <?php }?> </p> <?php } else {?> <a href="<?=htmlspecialchars("?thumb=$row->RowId")?>"> <img src="<?=htmlspecialchars("?thumb=$row->RowId")?>" title="<?=htmlspecialchars($row->Title)?>"> </a> <?php }?> <?php }?> </div> </body> </html>

Performance Tips

One of the main (and only) challenges in Stagsi is thumbnail loading. If you are browsing through a list of 1000 objects and every object’s thumbnail is 100 KiB (a modest estimate) – then Stagsi needs to load about 1000×100 KiB = 100 MiB of data really fast – and then to render it.

For traditional HDDs (spinning hard-disk drives) 100-150 MiB per second is the total limit they may provide in ideal conditions (only 1 process is using the drive, the file system is not fragmented, all files are near each other “physically”).

Modern SSDs and especially NVMe’s are inexplicably better at the kind of tasks Stagsi needs but may still fall short in some circumstances (large thumbnail files, multiple processes accessing the disk simultaneously, real-time antivirus filters, low-end hardware, etc.).

Therefore it’s recommended to all users to consider the following factors contributing to smooth Stagsi browsing, in order of importance:

Portable Builds

Stagsi doesn’t have to be installed on the target system. Moreover, it can be configured to avoid writing to any locations outside of its directory.

Note: by default Stagsi is coming with a tiny automatic update program which does write to the registry and keeps logs and partially downloaded updates in the current user’s directory (%APPDATA%). If you want to avoid that, rename Sup.exe (so you can update Stagsi manually by running the renamed file) or delete it (and update by extracting new portable version’s archive from the website).

Aspect Explanation Portable considerations
Registry If the user has accepted the license agreement on the start-up then the EulaAccepted key is created under HKCU\Software\Soletude\Stagsi. Due to legal requirements, this key cannot be disabled.
Logs When Stagsi is launched, it starts writing logs even before database settings are read. The logs go to Stagsi’s folder Logs subfolder, or to %APPDATA%\Soletude\Stagsi\Logs if the former is not writeable (as is the case when it’s installed in Program Files). Ensure Stagsi can write to its Logs subfolder.
Database The location of the database is specified in DataPath key of Bootstrap.json’s in the Stagsi’s folder. By default and when it’s missing or bad, %APPDATA% is used. The value is relative to Stagsi’s folder. Set DataPath key of Bootstrap.json to Database (a subfolder of Stagsi).
Write access On launch, Stagsi tests if the database file is writeable and automatically enables /ro (read-only) mode if it isn’t. Stagsi won’t break if placed in a read-only location.

Using an External (USB) Drive

Stagsi cares little for where its data files are located. For portability purposes, the database can be placed on an external USB drive, possibly even with Stagsi binaries to avoid installing the program on the target system (see the Portable Builds section for details).

Common questions about using an external drive for Stagsi:

Version Numbers

Internally, Stagsi is an interface (hence the “-i”) to Stags (“Soletude’s Tagging System”) which is internally called “the library” (and Stagsi called “the host program”). Both parts have separate versions exposed in Windows file properties (Soletude.Stags.Library.dll and Stagsi.exe) and in the database’s Stags table (shown in the Database panel in the UI).

LibraryVersion indicates the version of the database schema and is used in the migration process. It consists of two numbers: major and minor. DLL properties also include build number but it is ignored for the purpose of determining if two schema versions are the same. This version does not appear outside of the database field and DLL properties.

HostVersion indicates the program’s “overall” version as major.minor.build number, used in different places, e.g. in the Copy command’s JSON. Given two host versions, if their build numbers are different but other components are the same then it’s essentially the “same” Stagsi version, else the version having a larger build number is more recent (thus nothing needs to be calculate based on major and minor numbers, they only need to be compared as is). HostVersion in the database is purely informational.

Loosely following semver.org, major is incremented whenever a significant change occurs, possibly breaking old scripts, plugins, etc.; minor is incremented routinely for each new release; build number is used for internal accounting purposes.

Alternative Software

Stagsi is not the only solution in the field of data management but we believe the competition has critical flaws and so we’re not afraid of talking about it. If you’re not happy with Stagsi – get in touch with us go.soletude.ca/stagsi/forum. If you know another alternative – let us know and we will evaluate it.

Database From Scratch

A database can be entirely auto-generated, without relying on Stagsi GUI or executables at all. This has many uses – one is data “scraping”, i.e. taking of data from an arbitrary resource in an arbitrary format and presenting it uniformly using Stagsi.

Freelancer.com

Most websites present their information differently even though the underlying data may have the same structure and then fail at providing adequate means to work with it.

Freelancer.com is a resource where you can create design contests (among other things); you get a listing of submitted works – a plain simple grid of images with few sorting options, no filtering, fixed page size, fixed thumbnail dimensions, practically no rating capabilities. A contest may see thousands of submissions and reviewing them (especially if you have other team members) becomes a painful experience.

This can be solved by importing results into Stagsi – gaining sharing (git), tagging and filtering, sorting, backing up and so on. First, open your contest’s page, sort it in some stable manner (e.g. by time submitted), open the Console (Ctrl+Shift+I), paste this script and run it (Enter):

Import/Freelancer.com grabber.js • Skip this example (#exSJ) • Browse on GitHub

var f = function () { document.querySelectorAll('#entry-cards-list figure img.ContestCard-image') .forEach(img => a.push(img.getAttribute('src')) var pages = document.querySelectorAll('.btn.Pagination-link') pages[pages.length - 1].click() var t = setTimeout(f, 1500) window.onunload = () => clearTimeout(t) } var a = []; f()

Once it walks to the last page, run this in the Console: a.join('\n') – it will output a list of thumbnail URLs. Copy/paste them into a text file and feed that file to any download manager, such as GNU Wget: wget -iurls.txt

Finally, import the downloaded images into Stagsi as usual.

It’s safe to repeat this process multiple times because thumbnails do not change and so already imported images are detected as duplicates (same data hash).

Unicode Table

Unicode is a standard for encoding human texts. It strives to encompass all writing system in use today, in the past and in the future (emoji) and so the number of symbols known to this standard is huge (tens of thousands). Each symbol is assigned a set of properties, such as “Latin plane” or “Right-to-left script”.

Below is a PHP script filling an empty database (copied from Skeleton folder) with Unicode symbols, creating thumbnails for each of them and assigning tags according to the symbol’s properties. Things to note:

Database/Unicode.php • Skip this example (#exYD) • Browse on GitHub • Adjustments (3)

<?php set_error_handler(function ($severity, $msg, $file, $line) { throw new ErrorException($msg, 0, $severity, $file, $line); }, -1); if (!class_exists('IntlChar')) { die('php_intl module is required.'); } copy('C:/Program Files/Stagsi/Skeleton/Stagsi.sqlite', 'Stagsi.sqlite'); copy('C:/Program Files/Stagsi/Skeleton/Settings.json', 'Settings.json'); $settings = json_decode(file_get_contents('Settings.json')); $settings->BitmapScalingMode = 'Fant'; $settings->ThumbFormat = 'png'; $perDir = $settings->FolderSize; file_put_contents('Settings.json', json_encode($settings, JSON_PRETTY_PRINT)); $bom = "\xEF\xBB\xBF"; $rootTagID = 1; $assoc = [ [ "CallMode" => 1, "ExecutableList" => [ "nircmd clipboard readfile \"%1\"", ], "Extension" => ".unicode" ], ]; file_put_contents('Associations.json', json_encode($assoc, JSON_PRETTY_PRINT)); $db = new PDO('sqlite:Stagsi.sqlite'); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $selectTag = $db->prepare('SELECT RowId, * FROM Tags WHERE Title = :title AND ParentRowId = :parent'); $selectRootTag = $db->prepare("SELECT RowId, * FROM Tags WHERE Title = :title AND ParentRowId = $rootTagID"); $insertTag = $db->prepare("INSERT INTO Tags (Title, CreationTime, \"Order\", ParentRowId, IconPosition) VALUES (:title, STRFTIME('%s','NOW'), 0, :parent, 0)"); $insertObject = $db->prepare("INSERT INTO Objects (CreationTime, Random, Hash, Title, Format, ThumbCropX, ThumbCropY, ThumbCropWidth, ThumbCropHeight, FileSize, Codepoint) VALUES (STRFTIME('%s','NOW'), RANDOM() % 2147483647, :hash, :title, :format, 0, 0, 0, 0, :size, :codepoint)"); $insertObjectTag = $db->prepare('INSERT INTO ObjectTags (DataObject_RowId, Tag_RowId) VALUES (:object, :tag)'); $ver = join('.', IntlChar::getUnicodeVersion()); $db->exec("INSERT OR REPLACE INTO Stags SET Key = 'Unicode', Value = '$ver'"); try { $db->exec("ALTER TABLE Objects ADD COLUMN Codepoint INT NOT NULL DEFAULT 0"); } catch (PDOException $e) { // Already added, ignore. } list($formatTagID) = createTag(['Unicode File']); $db->exec("UPDATE Tags SET System = '_unicode' WHERE RowId = $formatTagID"); $codepoints = []; // Generation of 0x0000..0xFFFF (55346 codepoints) takes 30 minutes. IntlChar::enumCharNames(0x0000, 0x0FFF, function ($codepoint) use (&$codepoints) { $codepoints[] = $codepoint; }); $count = 0; foreach ($codepoints as $codepoint) { if (++$count % 500 == 0) { printf('%s / %s (%.1f%%)...%s', $count, count($codepoints), $count / count($codepoints) * 100, PHP_EOL); } $text = IntlChar::chr($codepoint); $insertObject->bindValue(':hash', md5($bom.$text)); $title = IntlChar::charName($codepoint, IntlChar::UNICODE_CHAR_NAME); $insertObject->bindValue(':title', $title); $insertObject->bindValue(':format', 'unicode'); $insertObject->bindValue(':size', $bom.$text); $insertObject->bindValue(':codepoint', $codepoint); $insertObject->execute(); $objectID = $db->lastInsertId(); $objectGroup = floor($objectID / $perDir); linkTagsTo($objectID, $formatTagID); is_dir($objectGroup) or mkdir($objectGroup); file_put_contents("$objectGroup/$objectID.unicode", $bom.$text); drawText("$objectGroup/u$objectID.png", $text); $age = IntlChar::charAge($codepoint); $age[count($age) - 1] = 'Unicode v'.join('.', $age); createTagFor($objectID, array_merge(['Age'], $age)); $digit = IntlChar::charDigitValue($codepoint); $digit < 0 or createTagFor($objectID, ['Digit', $digit]); $num = IntlChar::getNumericValue($codepoint); $num === ((float) -123456789) or createTagFor($objectID, ['Numeric', $num]); $dir = IntlChar::charDirection($codepoint); createTagFor($objectID, ['Direction', prettyConst($dir, 'CHAR_DIRECTION_')]); $names = [ 'Name' => IntlChar::UNICODE_CHAR_NAME, 'Name/Alias' => IntlChar::CHAR_NAME_ALIAS, 'Name/Choice Count' => IntlChar::CHAR_NAME_CHOICE_COUNT, ]; foreach ($names as $tag => $const) { $name = IntlChar::charName($codepoint, $const); if ($name) { createTagFor($objectID, array_merge(explode('/', $tag), [$name])); } } $name = IntlChar::charName($codepoint, IntlChar::EXTENDED_CHAR_NAME); if ($name !== IntlChar::charName($codepoint)) { createTagFor($objectID, ['Name', 'Extended', $name]); } $cat = IntlChar::charType($codepoint); createTagFor($objectID, ['Category', prettyConst($cat, 'CHAR_CATEGORY_')]); $pair = IntlChar::getBidiPairedBracket($codepoint); $pair === $codepoint and $pair = false; $lower = IntlChar::tolower($codepoint); $lower === $codepoint and $lower = false; $upper = IntlChar::toupper($codepoint); $upper === $codepoint and $upper = false; if ($pair !== false or $lower !== false or $upper !== false) { $apath = "Attachments/$objectID"; is_dir($apath) or mkdir($apath, 0750, true); $pair === false or file_put_contents("$apath/Pair.unicode", $bom.IntlChar::chr($pair)); $lower === false or file_put_contents("$apath/Lower.unicode", $bom.IntlChar::chr($lower)); $upper === false or file_put_contents("$apath/Upper.unicode", $bom.IntlChar::chr($upper)); } $block = IntlChar::getBlockCode($codepoint); // E.g. block ID 279 has no constant. createTagFor($objectID, ['Block', prettyConst($block, 'BLOCK_CODE_') ?: "#$block"]); $class = IntlChar::getCombiningClass($codepoint); $class and createTagFor($objectID, ['Combining Class', $class]); $closure = IntlChar::getFC_NFKC_Closure($codepoint); strlen($closure) and createTagFor($objectID, ['FC NFKC Closure', $closure]); } echo "Wrote $count codepoint objects.", PHP_EOL; function createTagFor($objectID, array $path) { $ids = createTag($path); linkTagsTo($objectID, end($ids)); } function createTag(array $path) { global $rootTagID, $db, $selectTag, $selectRootTag, $insertTag; $ids = []; foreach ($path as $i => $title) { if (!$i) { $selectRootTag->bindValue(':title', $title); $selectRootTag->execute(); $row = $selectRootTag->fetch(); } else { $selectTag->bindValue(':title', $title); $selectTag->bindValue(':parent', end($ids)); $selectTag->execute(); $row = $selectTag->fetch(); } if ($row) { $ids[] = $row['RowId']; } else { $insertTag->bindValue(':title', $title); $insertTag->bindValue(':parent', end($ids) ?: $rootTagID); $insertTag->execute(); $ids[] = $db->lastInsertId(); } } return $ids; } function linkTagsTo($objectID, $tagIDs) { global $insertObjectTag; foreach ((array) $tagIDs as $tagID) { $insertObjectTag->bindValue(':object', $objectID); $insertObjectTag->bindValue(':tag', $tagID); $insertObjectTag->execute(); } } function prettyConst($value, $prefix) { static $consts; if (!$consts) { $consts = (new ReflectionClass(IntlChar::class))->getConstants(); } foreach ($consts as $cname => $cvalue) { if (!strncmp($cname, $prefix, strlen($prefix)) and $cvalue === $value) { return ucwords(strtolower(strtr(substr($cname, strlen($prefix)), '_', ' '))); } } } function drawText($file, $text) { $im = imagecreatetruecolor($w = 128, $h = 128); imagesavealpha($im, true); // Windows Explorer would show 100% transparent pixels as fully opaque. // E.g. if 127 is changed to 126, the image would appear truly transparent. // But it's an Explorer's quirk - Photoshop does show proper 100% transparency. imagefill($im, 0, 0, imagecolorallocatealpha($im, 255, 255, 255, 127)); $font = 'c:/windows/fonts/ARIALUNI.TTF'; $size = 48; $box = imagettfbbox($size, 0, $font, $text); // 64x64 // 1:7 --- 32:7 // | '@' | // 1:37 --- 32:37 $x = round(($w - $box[2] + $box[0]) / 2); $y = $h - round(($h - $box[1] + $box[7]) / 2); //if ($text==='@') { // foreach ($box as $i => &$ref) { $i % 2 and $ref += 30; } //var_dump($text,$box,$x,$y); //imagerectangle($im, $box[6], $box[7], $box[2], $box[3], imagecolorallocate($im, 255, 0, 0)); //} imagettftext($im, $size, 0, $x, $y, imagecolorallocate($im, 0, 0, 0), $font, $text); imagepng($im, $file); }