Fox Row

Titles on Netflix

2023-12-13T12:00:00-06:00

A word cloud generated from Netflix viewing data, January-June 2023

A histogram of Netflix watch time by show, January-June 2023

NaMoGenMo 2023 report

2023-12-02T12:00:00-06:00

Over the month of November, I worked on a project to create code that generates a movie.

My project this year: see most exciting parts of a movie right away, and leave all those pesky slow, boring parts for the end. Bottom Line Up Front.

What parts of a movie are the most exciting? The one with the most on-screen motion, of course! We can BLUF-ize a movie by extracting the segments with the most movement in the whole video. Then the next most action, and so on until you are left with the slowest, most sedate parts.

So whenever you choose to stop, everything you've already seen is more exciting than what's left.

As a subject, I chose The Phantom of the Opera (1925), a silent film starring Lon Chaney as the Phantom. It was released in 1925, meaning it's in the public domain.

The code I wrote to generate the video is available on GitHub.

I am very pleased with how it turned out! It's choppy, but more or less intelligible. And having all dynamic shots front-loaded is pretty exciting. The final result:

Thought about Cosmopolitan C

2023-11-05T12:00:00-06:00

The Cosmopolitan C project just released v3.0 of the library. It's a technical marvel - compile a single binary that runs on 6 OSes, across 2 architectures, and also on bare metal.

It occurs to me - since BIOS is supported, one could probably compile Firefox with cosmoc, and get the METAL operating system from Birth & Death of JavaScript.

If you haven't seem that talk, do yourself a favor and watch it, and then click around on the cosmo c project website.

Querying a power outage database

2023-07-16T12:00:00-05:00

I have been tracking power outage data using a technique I learned about from Simon Willison called git-scraping. The outages are broken down by city, a stylized version of the data looks like:

1pm     seattle: 300
2pm     seattle: 200    tacoma: 150
3pm                     tacoma: 140     # implied seattle = 0
4pm     seattle:  50                    # tacoma = 0, seattle has new outages

Mr. Willison also has written a library to convert a git-scraped repo into an SQLite database, called git-history.

Some simple information I'd like to query the database for: what's the total number of customers experiencing an outage over time? My previous solution parses the JSON in each commit in the git repo, but it's kind of slow, and I'd like to use git-history for more complex queries later on. However, I ran into two issues.

Data format

The first problem was my data format. If a city drops from the list of outages between two commits, that implies the outages there were fixed. (See the 3pm line in the example above.) By default, git-history interprets an entry disappearing from the data as "no change".

I solved this in a straightforward, brute-force fashion: at each commit, query the DB for all known cities. Set their outage count to 0, and then overwrite with any cities actually in the current commit.

Using the --convert option in git-history:

git-history file outages.db outages.json --id "name" --namespace "outage" \
--import sqlite_utils \
--import sqlite3 \
--convert '
from sqlite_utils import Database

db = Database("outages.db")

# get the name of every city in the DB so far
known_cities = {city["name"]: 0 for city in db.query("SELECT DISTINCT name FROM outage")}

# overwrite 0 for cities actually in this commit
for city in json.loads(content):
    known_cities[city["name"]] = city["outage_count"]

for city in known_cities:
    yield city
'

It's not pretty, but it works!

Querying the database

The second issue was actually querying the DB. I have technically found a solution... but it's terribly slow. 2 nested subqueries! 😬 This is the general idea:

for each timestamp there's a commit at:
    sum the outage counts for:
        select the most recent row for each city (not necessarily in the "current" commit!)

Here's the query I have:

-- for each commit timestamp
select
  distinct _commit_at,
  (

    -- sum the outages for a given datetime
    select
      sum(customers_affected) as total
    from
      (

        -- select the most recent # of outages for each city, on or before the given datetime
        select
          max(inner._commit_at) as timestamp,
          customers_affected
        from
          outage_version_detail as inner
        where
          inner._commit_at <= outer._commit_at
        group by
          _item
      )
  )
from
  outage_version_detail as outer

I believe it runs correctly, but at ~6 seconds per row returned, it would be faster to iterate over the original git repo and parse the JSONs. Surely there's a better way to structure the two inner queries, but I haven't gotten there yet. If anyone has ideas, please get in touch!

TIL: converting JSON using git-history

2023-06-23T12:00:00-05:00

git-history is a tool created by Simon Willison, that makes it easy to work with data collected via git scraping.

It expects data in a particular format - a flat JSON array. If your data isn't already there, you can use the --convert option to run any python code to make it the right shape.

For example, if your JSON looks like this:

{
    "incidents": [
        {
            "id": "552",
            "name": "Hawthorne Fire",
            "engines": 3
        },
        {
            "id": "556",
            "name": "Merlin Fire",
            "engines": 1
        }
    ]
}

You might ingest it via a command like this:

git-history file incidents.db incidents.json --id "id" --convert "json.load(content)['incidents']"

If you're using git scraping to track any data, it's useful for working with the data efficiently, so try it out!

APSE updates

2023-04-30T12:00:00-05:00

It has been a while since I wrote about APSE here, and over the past few months I've made a number of big updates to it in version 4.

Performance

First of all, there have been major performance improvements. Users can expect much faster (i.e. lower CPU usage) processing of snapshots. Along with that is an improvement to OCR accuracy.

Search parsing

Searches are more forgiving. For example, searching bike will match bikes, biker, biked, biking, etc. You can still specify exact matches "with quotes." URL components are also handled better - wikipedia will match anything with that in a web address, you don't need to specify all of en.wikipedia.org.

Archive stuff

PDF imports are now supported.

You can now merge indexes from multiple machines. (Business licenses only.)

Fun stuff

Latest snapshot: you can jump directly to the most recent snapshot taken, by using the special search latest or the Go menu in the app. You can still search for the word latest by combining it with another keyword, using quotes, or adding a space afterward: latest.

Starred snapshots: you can star snapshots you want easy access to. You can filter on them in searches using the format starred:true.

More Like This: find a snapshot that's not exactly what you're looking for? Click "More Like This" to see snapshots with similar content.

I think it's a really fantastic app, and this round of updates makes it even better. So if you've read this far, I encourage you to try it out!

TIL: API for saving webpages in the wayback machine

2023-04-08T12:00:00-05:00

The Internet Archive's Wayback machine is a way to save a copy of a public webpage at a point in time. There is an API to submit URLs for them to crawl, but the documentation I found is fragmented and some is out of date. Here is what I have got working:

First, you need an account with archive.org. Some of the documentation you can find via Google contradicts this, but I've tried with and without credentials, and unauthenticated doesn't work for me. Signing up is free.

Once you have an account, log in. You need to generate S3 credentials (as in AWS? It's unclear to me). You do that at https://archive.org/account/s3.php. There's two parts to the credentials, an access key and secret key. Both are a medium-ish length random string. These will be used for authentication, via a request header later.

Now that you have credentials, you can submit save requests for webpages on the wild internet. You do so by submitting an HTTP request to https://web.archive.org/save/. The full request I used looks like this in cURL:

curl -X POST -H "Accept: application/json" -H "Authorization: LOW <access key>:<secret key>" -d"url=https://example.org/url-to-save&capture_all=1&delay_wb_availability=1&skip_first_archive=1" "https://web.archive.org/save"

Explanation

curl -X POST [...] "https://web.archive.org/save": we are submitting an HTTP POST request to archive.org via the command line, using cURL.

-H "Accept: application/json" -H "Authorization: LOW <access_key>:<secret_key>": Headers to put in the request. The Authorization header is required, as far as I can tell, despite some documentation you can google to the contrary. Replace <access_key> and <secret_key> with your credentials. Note the colon between them. Don't include the literal < or > characters in your request. Specifying a JSON response is not required.

-d"url=https://example.org/url-to-save&capture_all=1&delay_wb_availability=1&skip_first_archive=1": the payload for the request. url is well, the URL you want to save. capture_all=1 tells the crawler you want to capture all HTTP responses, even 4XX or 5XX errors. Without this, only 200 OK responses are saved. delay_wb_availability tells Wayback that you don't need the page saved immediately. If the crawler is overloaded, it will go onto the end of the queue for indexing when load is lower. skip_first_archive tells Wayback to skip checking if this is the first time the URL has been archived. They say it increases archiving speed, so I went with it.

How I got there

This stack exchange answer indicates that the Save Page Now API is how you can programmatically submit URLs for saving. I kind of had to google around for a while, and eventually stumbled on the info that "Save Page Now API" is the magic words. There are other IA API docs, but don't mention anything about Save Page Now that I can find.

That stackexchange answer links to this documentation in Google Docs (curious to me that it's not on an archive.org URL, or the existing IA API docs). That page links to more Google Docs with the actual API specification.

Further research

The page to manually submit a save request to Wayback has an option to "Save also in my web archive". Your user account has a (public!) list of pages you have personally saved.

It seems the form on that page POSTs to the same web.archive.org URL. As best I can tell, after you submit the form, the webpage waits for the indexing request to begin, receives some job ID from Wayback, and then submits another request using that ID and your account info. I have been unable to get this working with the Save Page Now API via any combination of the name field corresponding to that checkbox.

If anyone knows if saving to My Web Archives is possible via API, please get in touch!

video-to-sqlite

2023-01-21T12:00:00-06:00

Inspired by Simon Willison's collection of sqlite tools, I created a library for extracting information from a video, and storing it in an sqlite database. It's called video-to-sqlite. You can also install it via pip install video-to-sqlite.

You can use Simon's very nifty tool Datasette to explore the resulting DB. Let's use this exploration of John Green's powers of discernment with regard to soda brands:

Create the database like so, and begin serving it locally:

video-to-sqlite vlogbrothers.db green.mp4
datasette vlogbrothers.db

If we visit localhost:8001 in a browser, we can see there's a little over 9,000 frames in the video, and metadata about each:

We can also run custom sql against the database. For example, counting how many keyframes, predicted frames, and bidirectional predicted frames there are:

This is already pretty interesting. However, we can do even more with this.

Creating a custom search engine for video contents

video-to-sqlite supports running a custom function against every frame during extraction. One could use that hook to OCR the frame and store that along with the rest of the metadata, for example. You might write that something like this:

import video_to_sqlite

def ocr_callback(frame, metadata):
    text = ocr(frame)
    return {'text': text}

video_to_sqlite.main('vlogbrothers.db', 'green.mp4', prefix=None, callback=ocr_callback)

You would then get a db with those text contents associated with the frame:

But we can still do one better. Datasette has a plugin called datasette-media. It lets you serve static files alongside regular datasette responses. For example, viewing images associated with a URL, like the imageset used to train Stable Diffusion.

I hacked up datasette-media to return individual frames from a video on request. Combined with datasette, this can show you the frame in question for each row. You need to specify some config values for datasette, in a file usually called metadata.json:

{
    "plugins": {
        "datasette-media": {
            "video": {
                "sql": "select filename as filepath from frames where filename=:key",
                "enable_transform": true
            }
        }
    }
}

Now if you run datasette -m metadata.json vlogbrothers.db you can get something like:

Additionally, datasette also functions as a JSON API. Combining everything, we can create an API to grep video contents and get the corresponding frames:

curl "http://localhost:8001/vlogbrothers.json?_shape=arrayfirst&sql=select+%22http%3A%2F%2Flocalhost%3A8001%2F-%2Fmedia%2Fvideo%2F%22+||+filename+||+%22%3Fframe_no%3D%22+||+frame_no+||+%22%26w%3D200%22+as+frame+from+frames+where+pict_type+%3D+%22I%22+order+by+filename%2C+frame_no+limit+101"

[
  "http://localhost:8001/-/media/video/green.mp4?frame_no=0&w=200",
  "http://localhost:8001/-/media/video/green.mp4?frame_no=71&w=200",
  "http://localhost:8001/-/media/video/green.mp4?frame_no=94&w=200",
  "http://localhost:8001/-/media/video/green.mp4?frame_no=188&w=200",
  "http://localhost:8001/-/media/video/green.mp4?frame_no=300&w=200",
[...]

Sound useful? Interesting? Let me know if you use it for something cool.

Video from Vlogbrothers used under CC-BY

Remapping keys in windows

2022-12-19T12:00:00-06:00

Today I learned about Powertoys,, a utility application from Microsoft.

Caps lock isn't a very useful key, but it occupies very valuable keyboard real estate, sitting on home row. Some programmers like mapping ctrl to the caps lock key. Some keyboards even have hardware settings to switch them. Personally, I find it awful for trying to hit ctrl-z, x, c or v.

But the escape key, I hit a lot. I've known MacOS allows mapping escape to caps lock, but the last time I looked for windows, it seemed like everything was registry hacks and shady applications.

But with powertoys, I feel good that it's an officially supported, first-party app. And it lets you map arbitrary keys, not just a few modifiers. Powertoys offers a lot other features (the stay-awake one seems like I'll use it on my laptop), but the keyboard remapping alone is good enough to get me on board.

Disabling the extensions button on Firefox toolbar

2022-12-13T12:00:00-06:00

In Firefox 109, Mozilla introduced a "unified extensions button". Normally, you can fully customize your toolbar, but this button cannot be moved or removed from the toolbar via the usual "Customize Toolbar" method.

However, it can be removed with an edit to your Firefox config. To remove it, open about:config. There is a boolean setting extensions.unifiedExtensions.enabled that defaults to true. Change it to false, restart Firefox, and you shouldn't see it in your

Hopefully Firefox makes it normally customizable in the future, like every other item on the toolbar.

Thanks to "Unified extensions toolbar button" in this Reddit post.

Crossfading in MoviePy

2022-12-05T12:00:00-06:00

Taking a cue from Simon Willison, a TIL (today I learned) about the moviepy library.

The library is very useful, but sometimes I have difficulty finding documentation on how to do specific things. For example, recently for NaMoGenMo I wanted to crossfade between 2 clips.

Suppose you have two 10-second video clips, and you want to cross-fade from clip A to clip B over 3 seconds. So clip A starts playing at 00:00, starts fading out at 00:07, and is completely faded out at 00:10. Clip B starts playing at 00:07, immediately starts fading in, and is finished fading in at 00:10. How you could accomplish that:

overlap = 3
final_clip = CompositeVideoClip([
    clip_a.crossfadeout(overlap),
    clip_b.set_start(clip_a.duration - overlap).crossfadein(overlap)
])

NaMoGenMo 2022 report

2022-11-30T12:00:00-06:00

This November, I worked on a project to create code to generate a movie. I got the idea from NaNoGenMo, which itself is based on NaNoWriMo.

My idea was to create a Memento-ized version of an existing movie. If you haven't seen Memento, it's structured where the order of scenes is interleaved, so the beginning of the movie is the last thing to happen to the characters chronologically.

If the order of events happening to the characters is A, B, C... Y, Z, then what you watch is Z, A, Y, B, X, C, etc etc. The end of the movie you watch is when the backwards and forwards scenes meet in the middle. The first-half scenes are in black and white to help the audience a little.

To accomplish this in a generative fashion, I wrote a library to assist me. It's called mementoizer, and it's available on PyPI.

I wanted to use it on a color film to take advantage of the alternating color/black and white structure. Unfortunately, there are not a lot of color feature films in the public domain/permissively licensed. (I do see a few places that indicate the 1952 version of The Snows of Kilimanjaro is in the public domain in the US, but I am not sure of the validity of those claims. If any copyright lawyers out there can clarify things, please get in touch as I think that would be an interesting movie to memento-ize.)

Given that constraint, and my desire to have something at least 40 minutes, I chose Memphis Belle: A Story of a Flying Fortress, a 1944 film produced by the US government.

I created the mementoized version with a minimum scene length of 3 minutes. If you install mementoizer, here is the command I used if you want to replicate the results:

mementoize memphis_belle.mp4 --min-scene-length 180

It's not human-perfect edited, but I am very pleased with how well it worked out! It obviously can't tell the difference between a shot cut, and a scene ending, but for a first cut, min-scene-length works pretty well. I think the result is effective and surprising in a good way. The final product:

Slay the Spire First Run Winrate

2022-11-26T12:00:00-06:00

A question came up in a discord for Slay the Spire - what is the win percentage of players in their first time playing the game? I thought that was an interesting question, so I looked into it using the big dataset from Mega Crit, the creators of the game.

Overall, players on a first-time run win about 9.6% of the time.

Details

My database is a subset of the full 77 million+ run set, containing mostly runs from after Watcher was released, when the game was near its current state. My database contains 18,209,495 runs total.

From there, I removed runs that are from beta builds, endless mode, daily mode, or chosen-seed runs. The vast majority of the remaining runs are from builds 2020-01-27 (Watcher update) and 2020-07-30 (the language patch).

From there, I counted runs where player_experience = 0. One thing to note is that this means the first time an account has played a run, not necessarily the first time a person has played a run.

Patch	Wins	Losses	Winrate
2020-01-27	2,069	32,357	6.0%
2020-07-30	8,357	76,709	9.8%
Overall	10,426	109,066	9.6%

The all-character winrate at ascension 0 is 9.3%, so I find it interesting that the "average" A0 player across all characters is a hair worse than the average player on their first run.

I think this is due to at least two things:

The first run is simpler, in that there are less card options in play, and the boss events are scripted.
The first run is with Ironclad, which has a higher A0 winrate than the other characters.

The Ironclad winrate at A0 is 10.7%, so at least that right-side up.

Questions

Winrate difference

The winrate difference between the builds is surprising to me - I think the language patch didn't affect gameplay. If the game hasn't changed, this is very unlikely to have occurred by chance. A chi-squared test puts the odds of that difference occurring randomly at less than .001% (p-value < 0.00001)

Non-Ironclad runs

Normally, the first run when you play the game is with Ironclad, since none of the other characters are unlocked yet. However, the dataset contains 337 Silent runs, 105 Defect runs, and 60 Watcher runs where player_experience = 0.

Other builds

My dataset only encompasses post-Watcher Spire. The consensus is that for high-level play, the game has gotten easier as patches and tweaks have made their way into the game. It would be interesting to see if the winrate for beginners has changed over time as well.

If you have any ideas, please get in touch to let me know!

Summing an array in SQLite

2022-08-30T00:00:00-05:00

Say you have a table with an array column in SQLite, and want to calculate the sum of that array for each row. Suppose your table is named my_data in SQLite and looks like:

id  |   my_list
-----------------
1   | [ 1, 2, 3 ]
2   | [ 4, 5, 6 ]
3   | [ 7, 8, 9 ]

You can sum up the values in the my_list column via the following SQL:

select t1.id, sum(t2.value) from my_list as t1 join json_each((select my_list from my_data where id = t1.id)) as t2 group by id order by id;

It should return the following:

1 | 6
2 | 15
3 | 24

Hope it saves you some time.

SGDQ 2022

2022-08-16T00:00:00-05:00

During a Games Done Quick marathon, runners, and commentators... commentate on the game currently being run. Commentary for different games has different character - some jokey, some focused on technical explanation, some rapid-fire play-by-play.

I wanted to see if you could get a flavor of the commentary for each run from a visualization, so I generated a word cloud from the transcript of each run.

A word cloud for each game from SGDQ 2022, in run order:

Shadow of the Colossus by RUBIEHART in 47:29

Sonic Generations by Allegro in 54:21

Warioware: Touched! by Mr_Shasta in 31:31

Rolled Out! by Helix in 20:26

Mass Effect by MikeWave in 1:30:02

Kirby and the Forgotten Land by Mr_Shasta in 1:51:20

Pokémon Snap by quo in 30:31

Spyro the Dragon by Dayoman in 1:29:10

Perfect Dark by Zero_IF in 51:12

Crash Bandicoot 2: Cortex Strikes Back by Potty in 1:14:49

Perspective by AChocolateOrange in 11:40

Super Dream Dasher by Bullets in 28:05

Warden: Melody of the Undergrowth by Jaxler in 10:54

Super Bonk by LeadOstrich in 25:22

Bulk Slash by Aquas in 17:45

Alien Cabal by tbcr in 6:09

Bubble Bobble Part 2 by JaekRock in 27:24

Blaster Master by UraniumAnchor and davidtki in 45:31

Ninja Gaiden by The Retro Runner in 11:44

Altered Beast by JaekRock and Space Coast Gaming in 6:15

McDonald's Treasure Land Adventure by Lizstar in 21:32

NiGHTS into dreams... by pochilin in 23:10

The Pathless by MaeTheDKC in 1:52:40

Ty The Tasmanian Tiger by KamaCrimson in 29:20

Super Castlevania 4 by Crunan and DrunkenDraconian in 41:38

Castlevania: Harmony of Dissonance by JupiterClimb in 40:50

FAITH: Chapter III by Vynn in 9:48

Sly 2: Band of Thieves by Nave357 in 27:38

The Legend of Zelda: Link's Awakening (2019) by TGH, Glan in 1:40:32

ICO by SanchoPanda in 1:37:05

Grand Theft Auto: Vice City by KZ_FREW in 58:13

Phasmophobia by Brossentia, NPC, FlannelKat and Peace Egg in 29:17

Silent Hill 4: The Room by AnEternalEnigma in 1:03:38

Five Nights at Freddy's: Security Breach by Ace Delusional in 1:05:30

Outlast by sharkhat87 in 37:15

OMORI by starsmiley in 2:44:22

Greak: Memories of Azur by SaberaMesia in 34:00

Infernax by DanTheVP in 51:52

Majuu Ou by Omnigamer in 20:55

KAMIKO by cicada in 22:46

Earnest Evans by ZELLLOOO in 9:07

Ninja Warriors by Mega RetroMan in 29:47

Battle Axe by Jimmy_Diamonds in 10:11

Gunvolt Chronicles: Luminous Avenger iX 2 by Benja in 33:19

Mega Man Powered Up by WhiteHat94 in 31:43

Mega Man Xtreme 2 by FocusSight in 37:55

Mega Man: The Wily Wars by The Blacktastic in 31:49

Mega Man 5 by Ppotdot1 in 37:14

Yakuza: Like a Dragon by Froob in 4:09:52

Star Wars: Jedi Knight II - Jedi Outcast by CovertMuffin in 46:31

Halo Infinite by WaifuRuns in 1:26:12

Super Mario Odyssey by -ganon- in 3:21:30

Mario Party by duck in 39:15

Final Fantasy VI Randomizer by drewlith, DoctorDT, seto kiaba and Jexvrok in 1:44:23 - SGDQ 2022

Klonoa: Door to Phantomile by amoser in 42:43

The Lord of the Rings: The Return of the King by Maxylobes in 1:38:30

Pac-Man World 2 by Katie4 and McKiddy in 45:37

COGEN: SWORD OF REWIND by DemonchildElise in 44:52

PowerSlave Exhumed by Kaos_Wulf in 28:23

Knuckles' Chaotix by Hibnotix in 58:19

Sonic Adventure DX: Director's Cut by Katie4 in 10:07

Sonic Advance by Kirbymastah in 19:03

TUNIC by Sunnymuffin in 49:28

Advance Wars by legrandgrand in 54:28

Pokémon Emerald by Keizaron, 360Chrism, Shenanagans, adef in 3:14:04

Donkey Kong Country: Tropical Freeze by spikevegeta in 1:27:30

Minecraft: Java Edition by Illumina in 53:32

Super Mario Sunshine by SB_runs in 2:59:24

Banjo-Tooie by duck in 36:25

Devil May Cry by WaifuRuns in 1:31:13

DEEEER Simulator: Your Average Everyday Deer Game by Shockwve in 21:20- Summer Games Done Quick 2022

Mi Scusi by Bullets in 13:43

Turnip Boy Commits Tax Evasion by EldritchWolfie in 34:06

Jimmie Johnson's Anything With an Engine by Ghoul02, GhostKumo in 16:49-Summer Games Done Quick 2022

Gourmet Warriors by Ritzblues in 23:56

Thunder In Paradise by sharif in 34:04

Incredible Crisis by bramhallthefifth in 55:19

King's Quest V: Absence Makes the Heart Go Yonder! by davidtki in 27:46-Summer Games Done Quick 2022

The Little Mermaid by SteveZero01 in 7:08

Metal Gear by JosephJoestar316 in 23:22

Lumione by Breakdown and Elipsis in 38:00

Kirby Tilt 'n' Tumble by swordsmankirby in 42:41

Xenoblade Chronicles 2: Torna by Enel in 2:40:53

Slime Rancher by MaddyInc in 16:36

Solar Ash by bryonato in 1:03:27

Monster Hunter Rise by JalBagel in 1:09:29

StepMania_NotITG by SpootyBiscuit and WinDEU in 43:33

SOUND VOLTEX EXCEED GEAR by leviern in 1:19:00

Celeste by GrosHiken in 56:18

Borderlands 2 by Deceptix_ in 1:02:59

Doom Eternal by MuteD6 in 2:00:20

Control by DemonicRobots in 1:36:04

Liminal Ranger by Ozmourn in 10:07

Happy's Humble Burger Farm by teddyras in 29:30

Alisa by juh0rse in 52:04

Ato by Glitchiness in 24:37

Magic Trick by Biglaw in 7:54

Shovel Knight: Pocket Dungeon by DanTheVP in 13:56

Satisfactory by Epiphane in 22:38

Chocobo GP by Obiyo and groggydog in 32:32

Pokémon Shining Pearl by eddaket in 3:40:13

Donkey Kong Country by Tonkotsu in 29:38

F-Zero GX by Artful Hobbes in 29:42

The Legend of Zelda: Ocarina of Time by spikevegeta,Nukkuler in 2:25:23-Summer Games Done Quick 2022

Portal 2 TAS by Can't Even and mlugg in 47:13

Ocarina of Time TAS by dwangoAC, TASBot, Savestate, Sauraen in 53:05

Celeste.SMC by MrMightyMouse_ in 22:20

Super Mario World by Barbarian, juzcook, CarlSagan42, shovda, Shoujo, ThirdWall in 1:23:00- SGDQ2022

Ratchet and Clank: Going Commando by Xem in 1:15:48

Left 4 Dead 2 by WaifuRuns, aciidz, RsKiller and JurasPatryk in 50:04

Final Fantasy X by FoxyJira in 3:53:48

Metal Gear Solid 2: Sons of Liberty by dlimes13 in 1:11:52

Super Meat Boy by shredberg in 1:07:14

The Legend of Zelda: The Wind Waker by CLG Linkus7 in 2:47:57

Super Mario All-Stars + Super Mario World by SethBling, IsoFrieze in 11:55 - SGDQ 2022

Metroid Dread by kekumanshoyu in 1:20:33

Kaizo Super Metroid by Oatsngoats in 2:28:19

Super Mario Maker 2 by PangaeaPanga, juzcook, CarlSagan42, Aurateur, Shoujo, TanukiDan - SGDQ2022

Super Mario 64 by Simply in 1:40:17

Pokémon Brilliant Diamond by eddaket in 17:34

Elden Ring by catalystz in 1:53:24

Elden Ring by HYP3RSOMNIAC in 33:58

Non-run time. Runner intros, interviews, time between runs, etc.

NaMoGenMo

2022-06-23T12:00:00-05:00

You may have heard of NaNoWriMo, aka National Novel Writing Month. It's a challenge where every November, participants work to write a novel, defined as 50,000 words of fiction.

You may not have heard of NaNoGenMo, National Novel Generation Month,, where participants write code to generate 50,000 words of fiction. They define "novel" very loosely:

The "novel" is defined however you want. It could be 50,000 repetitions of the word "meow". It could literally grab a random novel from Project Gutenberg. It doesn't matter, as long as it's 50k+ words.

The idea occurred to me that the same is possible for video. "Movie" even works in the abbreviation - and so now we have NaMoGenMo.

The Academy defines a feature-length film as at least 40 minutes, so that is the definition I'm using for movie. (Though this is art, so it's hard to "do it wrong" - you could elect to create any length you want. Maybe we end up with a Short Film category as well.)

I am planning to run it during November, since that is when the other two Mo's happen. It also gives time to prepare.

As with the other events, there's really only one rule - that you share at least one movie and also your source code at the end.

I have created a repo for the 2022 edition. You can use Github's issues to track progress, register your participation, ask for help, etc.

So this November, you are invited to join me in moviemaking. Break a leg.

Solr and WSL

2022-06-21T12:00:00-05:00

Recently, I was trying to use Apache Solr on my WSL install of Ubuntu 20.04. Following the docs, I could download and start Solr, but visiting 127.0.0.1:8983/solr/ from a browser in Windows gave me an "unable to connect" error. The solr logs showed no errors, and the access log didn't even show an incoming request. curling the same URL from inside WSL gave a normal response.

It turned out WSL was not opening the port to localhost on Windows. To fix it, specify the command line argument -Dsolr.jetty.host. For example, to listen on all interfaces:

solr-9.0.0$ bin/solr start -h 0.0.0.0 -Dsolr.jetty.host=0.0.0.0

If you prefer to not listen on all interfaces, you can also use the internal WSL IP address, in the 172. range. This can be found in WSL in the result of ip addr.

To verify it worked, look for 0.0.0.0 (or the IP/hostname you specified) in the Solr log with something like, o.e.j.s.AbstractConnector Started ServerConnector@0a1b2c3d{HTTP/1.1, (http/1.1, h2c)}{0.0.0.0:8983}. You can also check Resource Monitor in Windows, in the Network tab, under "Listening Ports". Scroll down to 8983 and you should see your process.

Neither the Solr 9 getting started documentation nor the command line reference mention this flag. They do list the -h flag for setting a hostname, but it doesn't appear to be sufficient? I ran into the issue because I was using WSL, but surely there are Solr deploys in the wild that need to set a hostname. Anyone with inside information feel free to update me.

Hope it saves you some time.

Written for:

Windows 10
Ubuntu 20.04 on WSL 2
Solr 9

vid2emoji

2022-05-16T12:00:00-05:00

Video quality on computers is getting better all the time. Higher-density displays, more bandwidth, better codecs.

But what if you wanted video made out of emojis? Your wait is over - enter vid2emoji:

It’s a tool to convert videos to… ok they’re still videos but now it’s made out of emojis.

You can download it from pypi via pip install vid2emoji. Once it's installed, you can use it with vid2emoji <your video file>.

The source code is also available to check out on GitHub. If you put it to use, share what you make!

Yesterday's Wordle

2022-04-02T12:00:00-05:00

Update: The New York Times has an editor now who sometimes hand-selects words, e.g. FEAST on Thanksgiving. Given the frequent changes, this list is now unmaintained, and the word above is probably incorrect. Feel free to continue using it if you want a random-ish word for your first guess.
That word ☝️ is yesterday's Wordle answer. Use it as your first guess for an extra challenge.

This page updates at midnight your local time, so you always see yesterday's word.

Videosplatter

2022-02-28T12:00:00-06:00

The visual look and feel of movies is something that interests me. So I built a tool to visualize it.

Videosplatter is a tool to splat a video file into a mosaic of individual frames. It shows you at a glance the look and feel of the entire video.

You can see color palettes, light and shadow, and scene transitions in a single image.

It can generate mosaics:

Or "movie barcodes":

It also can generate "colorfulness" and contrast stats about the video. Colorfulness is the definition from Hasler and Süsstrunk 2003.

Add up colorfulness and contrast scores to get a "Visual 🌶️ Factor". Now you can complain about bland, blue-gray, washed out movies with science!

You can install it with pip install videosplatter. Let me know if you use it for something cool!

City footage used under CC-BY license from The Dronalist

Missed shot leaders in the NBA

2021-02-18T12:00:00-06:00

It's widely known that Kareem Abdul-Jabbar is the all-time leading scorer in the NBA, and LeBron stands a good chance of passing him when all is said and done.

What I've never seen before, is who is the all-time missed shots scoring leader? As in, who has missed the most points worth of shooting?

		Missed	Scored	Net
1	Kobe Bryant	34,314	33,643	-671
2	LeBron James	31,566	34,985	3,419
3	Dirk Nowitzki	29,357	31,560	2,203
4	Elvin Hayes	29,264	27,313	-1,951
5	Vince Carter	29,200	25,728	-3,472
6	Karl Malone	28,990	36,928	7,938
7	John Havlicek	28,054	26,395	-1,659
8	Carmelo Anthony	27,897	26,794	-1,103
9	Kareem Abdul-Jabbar	27,549	38,387	10,838
10	Wilt Chamberlain	27,437	31,419	3,982
11	Michael Jordan	27,332	32,292	4,960
12	Allen Iverson	26,995	24,368	-2,627
13	Paul Pierce	26,927	26,397	-530
14	Dominique Wilkins	26,179	26,668	489
15	Ray Allen	25,754	24,505	-1,249
16	Julius Erving	25,215	30,026	4,811
17	Elgin Baylor	24,584	23,149	-1,435
18	Rick Barry	24,277	25,279	1,002
19	Jamal Crawford	24,180	19,419	-4,761
20	Moses Malone	23,867	29,580	5,713

Through the beginning of this season, LeBron is averaging 1,815 missed points per year. At that rate, he will pass Kobe next season.

Active players in the top 50:

		Missed	Scored	Net
2	LeBron James	31,566	34,985	3,419
30	Russell Westbrook	22,560	20,759	-1,801
43	James Harden	21,146	21,549	403
50	Kevin Durant	19,831	23,491	3,660

Surprises

I would have guessed that Kareem was the leader - it takes so many field goal attempts to score that many, you're going to have a lot of missed shots too. But not having taken a lot of 3-pointers really bails him out. They're lower-percentage shots, and worth more, so you're in double jeopardy if you take a lot of threes.

Missed-shot-Kobe would be 4th on the regular scoring list.

I was surprised with how many guys were underwater: 23 out of the top 50. Since the NBA/ABA merger, the player deepest in the hole is #23 Jason Kidd, at -5,711.

Kareem is the leader in net score, and it seems unlikely anyone is going to catch him, at least without the game changing dramatically - say, different three point rules.

Information courtesy Basketball-Reference.com. Stats reflect games played through February 17, 2021.

Slay the Spire statistical analysis

2020-12-14T12:00:00-06:00

Slay the Spire is a single-player strategy card game I enjoy. In it, you build a card deck to defeat successively harder enemies as you move up levels in the Spire.

I recently learned that they released a huge data dump of runs from real-world play.

The data set

It's huge. 380 gigabytes huge. 75 million+ runs from October 2018 to November 2020.

I am only analyzing the 2020 segment of the data. Reason 1: the devs have made some changes to the game over time - balance tweaks, removed relics, that kind of thing. Reason 2: the 4th character, Watcher, was introduced early in 2020.

The 2020 data is fully after Watcher was introduced. It is still 18 million runs, totaling 90 gigabytes of raw JSON. I dumped it into an sqlite database, and it's still 77GB. On my SSD, running a count() query takes 6 minutes.

Reason 3: I don't need to deal with 4 times that.

Findings

Spire is hard. Out of the 18 million runs, there are 1.6 million victories, for an overall winrate of 9%.

Play speed

Which character plays the fastest? The slowest? Counting wins only,

Character	Avg. Time
Ironclad	58:32
Silent	1:09:11
Defect	1:06:44
Watcher	1:07:36

Interesting that Ironclad is so much faster, given the other 3 are so close.

The average win takes 64 minutes. The average loss takes 23. Well, of course if you stay alive longer, it takes... longer.

Spire has something called Ascension mode - extra levels of difficulty after you beat the game with each character. The game out of the box is Ascension 0. It goes up to Ascension 20.

We see that across the board from A1 to A20, runs between 60 and 80 minutes have the highest winrate. Notice the light line below the main cluster - that's Ascension 0.

This is putting the cart in front of the horse somewhat - it's not as if playing faster makes you better in a turn-based game. Rather, the easy wins tend to take around an hour, and the ones with maybe a narrower path to victory, requiring more planning and calculation, take longer.

Suppose you reach a particular floor, what are your chances of winning?

The discontinuities at floors 17 and 34 are due to the boss fights at the end of act 1 and 2 - if you beat the boss, you get some powerful rewards, and you're likely substantially better off than you were 1 floor ago.

Floors 50+ have some weird stuff going on - there's actually 4 different floors you can win on. One way is just beating the boss on floor 50. That accounts for the big spike on floor 51 - it's the "victory floor".

On Ascension 20, instead of fighting just one boss on floor 50, you fight another on floor 51, so floor 52 is the victory floor.

Once you unlock Ascension mode, you have the option throughout a run to collect 3 keys. If you do by the 50th floor, instead of winning the run then, you proceed to Act 4. On Ascensions 0-19, you end up fighting the Heart on floor 55. If you win, your victory floor is 56. If you fought two bosses (A20), your victory floor is 57.

Unfortunately, there isn't really a way to filter these out. A run isn't committed to Act 4 until you collect that last key, so it's possible to bail up until floor 49. A player may have been intending to collect all 3, but some bad luck or forgetfulness looks the same as someone who was just playing for floor 50 the whole way.

Deadliest Enemies

Which enemies are the toughest? Rather than just counting the number of kills by each one, let's take a look at how often they kill the hero.

Ironclad	Fatal%	Silent	Fatal%	Defect	Fatal%	Watcher	Fatal%
The Heart	51.12%	The Heart	47.64%	The Heart	52.75%	The Heart	51.96%
Automaton	35.87%	Time Eater	35.98%	Awakened One	33.22%	Time Eater	33.14%
Collector	31.75%	Awakened One	33.08%	Automaton	28.13%	Awakened One	27.71%
Awakened One	29.92%	Champ	32.85%	Collector	27.25%	Automaton	24.19%
Champ	29.28%	Collector	31.05%	Slaver and Parasite	24.24%	Collector	23.8%
Hexaghost	24.56%	Automaton	30.79%	Champ	23.95%	Champ	20.67%
Time Eater	23.53%	Slime Boss	30.65%	Time Eater	23.67%	Donu and Deca	19.9%
The Guardian	22.22%	Lagavulin Event	19.31%	Slavers	22.41%	Hexaghost	19.69%
Donu and Deca	21.23%	Donu and Deca	19.21%	Gremlin Leader	22.09%	Slavers	18.11%
Slime Boss	17.86%	Gremlin Leader	17.62%	Book of Stabbing	17.42%	Lagavulin Event	18.04%
Gremlin Leader	16.84%	Slavers	17.09%	Donu and Deca	17.04%	The Guardian	16.78%
Lagavulin Event	16.73%	Hexaghost	17.07%	Hexaghost	16.52%	Book of Stabbing	14.62%
Slavers	16.16%	The Guardian	14.77%	Masked Bandits	15.44%	Gremlin Leader	13.95%
Book of Stabbing	15.46%	Book of Stabbing	12.89%	Slime Boss	14.4%	Masked Bandits	11.3%
Slaver and Parasite	15.06%	Gremlin Nob	12.26%	Colosseum Nobs	13.85%	2 Orb Walkers	11.07%

The usual suspects are at the top - the Heart and act bosses are as deadly as you would guess, followed by a few elite fights. Seems crazy that Masked Bandits ends 10+% of Defect and Watcher runs!

That's enough for one round of analysis. If you find something cool in the dataset or have questions, let me know!

Shaq's finals streak

2020-11-16T12:00:00-06:00

I saw a statistic that at least one of Shaq's teammates has played in the NBA Finals every year for 37 years straight and counting. That sounds like a lot! For one thing, it starts 9 years before Shaq got in the league.

It got me wondering what the longest streak was. What else can we find out?

The longest streak

Shaq isn't the leader, believe it or not. Kevin Willis currently holds that honor, with 38 consecutive years with a teammate in the finals, spanning 1977-2014.

		Streak	Years active
1	Kevin Willis	38	1977-2014
2	Shaquille O'Neal	37	1984-
3	Toni Kukoc	36	1984-2019
4	Sam Cassell	35	1986-
5	Steve Nash	34	1987-
5	Wes Person	34	1984-2017
5	Tree Rollins	34	1974-2007
5	Tyronn Lue	34	1987-
5	Don Chaney	34	1957-1990
10	7-way tie	33

Shaq is well-poised to tie for the lead this season, given the number of active teammates he still has in the league (notably LeBron, who has gone to the finals 9 out of the last 10 years).

Most teammates in the finals

1	Shaq	201
1	James Edwards	201
3	Chuck Nevitt	186
3	A.C. Green	186
5	Richard Jefferson	184
6	LeBron James	182
7	Joe Kleine	176
7	Willie Naulls	176
9	Theo Ratliff	174
9	Corie Blount	174
9	Mel Counts	174

It looks like LeBron will move up this season. If he makes the finals, that adds ~15 to his total. Even if he doesn't, he has a lot of former teammates around the league. It seems likely that he will be #1 by the time all is said and done.

Surprises

Almost everyone who has played in the league has at least one finals teammate. Of the 4,814 players in the dataset, only 80 have no teammates who ever made the finals. Just 1.6%! Essentially every one is:

From the 40's/50's, or
only played a handful of career games (e.g. 10-day contract guys), or
are rookies/sophomores currently in the league.

At first I thought that can't be right. How has almost every rookie played with someone already? I realized that's thinking about it backwards - instead of rookies who haven't played with many teammates yet, think of every finals team as 15 guys (x2 teams) who can spread out across the league, and 1 finals player counts for a whole team.

For example, J.J. Redick went to the finals once with the Magic in 2009, and so every team he's been on since his rookie season in '06-'07 has at least 1 finalist. That's how Zion Williamson has a teammate, despite being a rookie who didn't even make the playoffs this year.

So who are these guys with no teammate-finals appearances? Some of the names that jumped out at me:

Ja Morant
Mo Bamba
Dennis Smith Jr. (only player with at least 3 years that I noticed)
Michael Porter Jr.
Jaren Jackson Jr.
Wendell Carter Jr. (what's with all the juniors?)

You get the idea. Players in their first few years in the league, who just happened to not have anyone on the team with them. I think this is basically accurate.

Caveats

There are obviously edge cases on what constitutes a "teammate" - for example, Memphis rostered Andre Iguodala the whole year up until the trade deadline, but he never played for them. Does he count for Ja Morant?

I decided to go with the rosters as defined on Basketball-Reference. So in that particular case, no. Iguodala is only listed on the Heat for the 2019-2020 season, he doesn't count for the Grizzlies.

It's possible this method under-counts some players who were traded, or gained or lost a teammate mid-season. If you find anyone notable, please let me know.

Information courtesy Basketball-Reference.com

Dark backgrounds, light backgrounds, and OCR

2020-09-27T12:00:00-05:00

One of my favorite parts to build in APSE was the way it handles light and dark backgrounds. Text recognition algorithms are tuned for black letters on a white background, like a printed page:

Usually this isn't a problem, because you can invert the colors if an image has light text on a dark background. This image also scans as "hello world".

But this won't work for Apse, because your screen might have both light and dark areas at the same time. For example, a dark-themed text editor open next to your browser. This image scans as "hello world igrelifemuZeyale!"

So Apse segments the image into dark and light regions, and selectively inverts the dark areas before converting to text. With that preprocessing out of the way, we don't get junk like "igrelifemuZeyale!".

If this was interesting, you can check out Apse in action at apse.io.

Shutting down projects

2020-08-19T12:00:00-05:00

Two of the older projects I have run for some time now are Needs More Lens Flare, a toy web app for automatically adding lens flare to photos, and Drone Stable, a more robust web application for automatically stabilizing aerial (or any kind of) video.

I am afraid I don't have the time to work on them any more. At the end of this month, I will be shutting both of them down.

I especially enjoyed working on and running Drone Stable, but right now I am more focused on Apse. If you haven't seen it before, check it out!

If you have questions or want to use software like that after the shutdown, please contact me.

Debugging dynamic querysets in Django

2020-03-14T12:00:00-05:00

Class-based views in Django are super handy. Among other things, you can create a view listing all instances of a model with about two lines of code.

One of the features of this ListView is the ability to customize the queryset used to populate the view. Suppose you wanted a view to list all the Foos in your database - no problem:

class FooView(ListView):
    model = Foo

Django also lets you customize the queryset, in case you don't want all Foos:

class FooView(ListView):
    """List all the Foos that aren't bar"""
    model = Foo
    queryset = Foo.objects.filter(bar=False)

I had to debug a problem with this. Django lets you specify a dynamic queryset by overriding the method get_queryset(). I was trying to do so by just overriding queryset. If you do that, Django will only evaluate it once, on startup. In my case, it was a time-based filter:

class FooView(ListView):
    """List all the Foos in the past"""
    model = Foo
    queryset = Foo.objects.filter(timestamp__lt=timezone.now())  # <- doesn't do what you think!

So my results would look right immediately after startup, but get more and more out of whack the longer the app ran. This ended up fixing my problem:

class FooView(ListView):
    model = Foo

    def get_queryset(self):
        return Foo.objects.filter(timestamp__lt=timezone.now())

Happy coding.

James Webb Space Telescope Countdown

2020-02-25T12:00:00-06:00

NASA's James Webb Space Telescope is the next big thing in space-based observatories. The project will enable never-been-done-before research, revealing astronomy and physics discoveries about dark matter, our galaxy, and more.

It has been in the news for delays in its launch schedule. If you ever need to know how long until it goes up, I created a tool for up-to-the-second updates on when it's going to launch. You can check it at (update: the countdown is retired, since it launched successfully on Christmas 2021. You can check out the telescope's current status at jwst.nasa.gov)

Apse 2.0 Release Announcement

2020-01-27T12:00:00-06:00

It's been a while since I have written about Apse here. I've still been working on it, and am happy to announce version 2.0 is now offically released! 😄

This is a substantial improvement over version 1. Both performance and general usage have seen big upgrades. I'm very happy with it! Here are the changes you'll see in v2.0:

The big features

💥 OCR is around 50% faster now. That means you can take snapshots more often, or keep the same frequency and see much lower CPU usage. This is thanks to both smarter preprocessing before OCR happens, and some performance gains in the OCR process itself.

💥 Dramatically better layout detection. In previous versions, the OCR algorithm read straight across the image left to right. If you had multiple windows open, or multiple columns in a document, the OCR'd text would be interleaved. Now regions of text are grouped much more intelligently.

💥 A visual indicator APSE is working during long running queries. Okay, it's a loading spinner. I've done some pretty convoluted advanced searches, and it's reassuring to see yes, your results ARE coming, be patient for once would you?

Smaller changes

Various UI improvements. The manual snapshot interface is clearer. There is also a much better indication of when your license key is expired, and how to fix it.

OSX Mavericks is no longer supported. Sorry. You'll have to upgrade to an OS released in the last three and a half years.

What now?

You can check out more information about Apse at apse.io or see a summary of the changes for all versions here.

I'm always happy to talk - if you have questions about it or any feedback, please get in touch! Happy searching.

Detecting Fire Engines in Imagery

2019-09-30T12:00:00-05:00

Finding fire trucks

One of the challenges in flying drones in public airspace is the need to peacefully coexist with everyone else. Emergency first responders like firefighters don't need quadcopters buzzing around, getting in their way.

Some consumer-level drones already have a "return to home" feature, where the drone automatically flies up to a safe height, returns to the spot it launched, and lands. This can happen when the battery is low, or the pilot hits a button.

What if we tried to automate returning to home when there is a fire engine nearby? For a first cut, we could simply try and detect if a fire engine is in the video feed from the drone. For our purposes here, we're not going to worry about where it is, if it's moving, how far away it is, what direction it's going, or anything like that. We're talking a pretty dumb autopilot, but if you're hanging around fires with your drone, maybe you need the help :)

So our decision process would be something like

See a fire engine? --> Land.
No fire engines in sight? --> Keep flying.

Image classification with ML would be a good place to start. Essentially, given a video frame or image, we want to classify whether the image has a fire truck in it. For training ML models, more data is usually better, but you don't need a ton to get started. I compiled a dataset of around 1,000 images with fire engines in them, and another 1,000 without. The fire engines came from the Imagenet dataset, and the non-fire engines from the Stanford cars dataset.

Keras is a machine learning toolkit that has some pre-built models vetted over large data sets. We could use InceptionV3, a neural net architecture from Google. Unfortunately, training on our 2k images doesn't give great results. After the net was trained, I picked a few images to measure predictions:

Predicted	Actual	Confidence
no	yes	52.6%
no	yes	52.6%
no	yes	51.6%
no	yes	52.7%
no	yes	52.2%
no	no	52.3%
no	no	51.6%
no	no	52.3%

Not good, and not confident! 50% is the same as random guessing. The poor quality was kind of to be expected - the loss measured by my validation image set never got below ~0.7. I suspect more example images would be needed to push that down.

How can we improve on this? Keras can also supply models with pre-trained weights on 1,000 Imagenet object classes. One of them is fire engines. Using Keras, we can get these pre-trained weights.

It's common to measure image classifiers by top-1 or top-5 accuracy. That is, what's the hit rate that the actual class is somewhere in the top 5 classes the algorithm predicts.

So how does InceptionV3 fare with pre-trained weights? Not good! In a sample of 5 fire truck images, here's what the out-of-the-box pretrained model predicts:

Rank	Confidence
#1	71.5%
#3	5.6%
Not in top 5
#3	1.1%
#1	18.8%

This confidence metric is a little different - it's split across 1,000 object types. So if the net predicts fire truck with 71% confidence, the other 999 add up to the remaining 29%.

This still isn't good. We do have another trick up our sleeve though. Since we only care about two classes - fire truck and not-fire-truck, we can re-train the pre-trained model with our data set. This is transfer learning, and the particular version we're doing is called fine-tuning.

We're going to take the original neural net and original weights, add some new layers, and train just our new layers. Once our new layers are "warmed up", we can tune the whole net to squeeze the last drops out of the model.

How well does it work? Glad you asked! With another sample of 5 positives and 3 negatives, the fine-tuned net gets all 8 correct.

Fire engine	confidence
yes	99.4%
yes	98.7
yes	85.0
yes	96.3
yes	91.6
no	94.4
no	79.8
no	84.3

On top of that, using the pre-trained weights decrased the training time per epoch from over 20 minutes to around 2 and a half. This was on an older desktop CPU as well - if you wanted to really juice the training speed, you could use a GPU for training.

Visualizing learning

You can plot the loss of the net over time, measured against a validation set of images, kept separate from the training set. Here's what that looked like for the nets I described here:

Next Steps

So we've found fine-tuning the pre-trained network weights for our specific goals improves accuracy dramatically. There's still a lot of room for improvement. Some ideas on how to make it better:

Recognize other emergency vehicle types, like ambulances or police cars. Those are both classes in the Imagenet 1,000, so given a moderate training sample I would expect you could see similar accuracy improvements.
Localize the vehicle in the image. This would let you determine relative positions of e.g. the drone and the emergency vehicle, and navigate safely away. This would mean using a different neural net architecture. R-CNN and family or YOLO would be a good place to start, especially YOLOv3.
More efficient neural nets. There is always a balance between accuracy, speed, processing power, and memory usage for these applications. Typically the processors on drones are relatively low-powered, and time is somewhat critical, so you want to be as lean as possible. On the other hand, you want to actually detect emergency vehicles. These pre-trained nets are unlikely to be the most optimal for any given hardware configuration. I briefly tried training the same dataset with MobilenetV2, another pre-trained model in Keras, but the results were very poor - it only got 2/8 correct, with confidences all over the place to boot.

I trained on my desktop machine, using CPU only. All training sessions pinned my CPU. That's good - it should be using all my cores. For the from-scratch training, RAM usage peaked just under 16GB. For the fine-tuning, it peaked around 10GB. I didn't look into why the difference - my guess would be tensorflow has some smarts about what layers are frozen/being trained. Fine tuning was also around 10x faster per epoch.

The code I used to train the net is available on Github. Hope you found this useful!

Memento Mori

2019-07-12T12:00:00-05:00

I have been working on Apse for about a year now. Up until a few days ago, I didn't know of any other software like it.

Earlier this week, Pinboard turned 10. It was one of my inspirations for Apse! Maciej's writing is gold, and his post to commemorate the occasion doesn't disappoint:

What does the future hold for Pinboard? Death! The bus that one day comes for us all! The skeletal, icy hand on an unprepared shoulder! Pain, a flash of light, then numbing darkness. So back up your bookmarks.

Just after I read that piece, I learned of a company called Atlas Informatics. In 2016, they raised $20 million in funding. The company folded later the same year. They were building... almost exactly Apse.

It was almost like whiplash - in the span of 10 minutes, I went from thinking my app was alone in the world, to learning of another one. Except that company raised 20 million dollars, ** and shut down less than a year later! ** And it all happened years ago.

The website is gone, but the press coverage and marketing materials are still around. It was a little surreal reading them - there's almost verbatim quotes of things I've been telling people about my project.

Companies and people come and go. Maybe I can learn from Atlas Recall. Their product was a little different - it had cloud features, and I have purposely decided against adding that to Apse. They only supported Mac OS, and Apse also runs on Windows and Linux. Apse still exists, which may be a point in its favor.

It does make me feel good that other people recognize the usefulness of this kind of application. If there's any users out there that wish they still could use Atlas Recall, have I got a deal for you! 🙂

So what does the future hold for Apse? Not death! Well yes, ultimately death. But hopefully not for a while. In the meantime, head on over to apse.io and check it out.

Apse 1.6 Update

2019-06-17T12:00:00-05:00

I have been able to add a lot of great stuff to Apse lately. Version 1.5 lets you manually trigger a snapshot from the system tray, along with many bugfixes.

Version 1.6 also has some great improvements. The UI has been made more clear in several places, and now you can update settings without restarting the app. This is great for the first time you boot it up, so you can enter the activation key and go.

I recommend everyone upgrade to the latest version - as of this writing, that is v1.6. Go ahead and get it at apse.io!

If you like Apse, you can really help by spreading the word. Tell anyone you think would find it useful - coworkers, friends, enemies, you get the idea. Post on social media if you're into that kind of thing. You can follow Apse on Twitter at @apseio.

Video Codecs: More Than You Wanted To Know

2019-04-25T12:00:00-05:00

How do video codecs work? How do they compress video into such tiny filesizes? A modern video codec is pretty complex, but you can get a pretty good idea of how they work at a high level.

What is a codec anyway?

Codec is short for COder/DECoder. Codecs aren't video files by themselves - codecs are just a part of one. Video files like MP4 are called containers. Each container can hold multiple audio and video streams. (Think different audio tracks for the same video.)

Today I'll just be talking about video codecs. The codec is what compresses the audio or video data, and decompresses it for playback. Each stream is separately encoded when the file is created, and decoded during playback.

Encode/compress and decode/decompress mean the same thing; I use them interchangeably here.

Just how good are they?

High-resolution video has a problem. It takes up so much space.

If you were to store the raw pixel data for HD video, you're going to run out of hard drives pretty quickly. Standard color video stores 8 bits per RGB channel, 3 channels per pixel, per frame.

Resolution	Framerate	Filesize
1080p	30fps	6 MB / frame	186 MB / second	11 GB / minute
4k	30fps	25 MB / frame	746 MB / second	44 GB / minute

Double those numbers for 60fps. So how can we avoid filling our phones after a minute of footage?

Take this stock video for example. It is in an mp4 container, encoded with the H.264 codec, 4k, 30fps, 11 seconds long. Raw bitrate from the calculations above would give 8.2 GB. The actual file is 30.6 MB, including audio. That’s 0.4% of the raw data.

How do they do it?

How can we get that crazy level of compression? Your first thought might be to compress each frame, a la JPG. Taking this frame as an example:

At high quality, it weighs 1 megabyte. At 30fps, this would be 30 MB/second, or 330 MB for the whole video. Still too high, 10x what the actual video file is.

So how do they actually work?

Codecs get such great compression by exploiting similarities between frames. Video frames are 1/30th (or 1/60th) of a second apart - they usually have a lot of overlap.

Video codecs store frames in 3 ways:

I frames, or intra-coded frames, also called keyframes. These contain full pixel data for the entire image.
P frames, or predicted frames. These don't carry full data for every pixel. They depend on the frame before, which may be a keyframe or another P frame.
B frames, or bidirectional frame. These are like P frames, but depend on the frame both before and after.

Video files contain framerate data, but that is more of a suggestion than a hard rule. What actually determines how fast frames are shown is their timestamp. Timestamps are calculated from data in the video file called timebase and PTS, short for Presentation TimeStamp.

Calculating the timestamp for a given frame is a bit complicated. Containers have a piece of data called the timebase. This is often a number like 1/90,000, for 1/90,000th of a second.

Each frame also has a PTS, which indicates it should be Presented at that timebase. So your first few frames in a video might look something like

Frame	PTS	Timebase	Time to display
1	0	1/90,000	0 * 1/90,000 = 0 seconds
2	3,000	1/90,000	3,000 * 1/90,000 = 0.033 seconds (1/30th!)
3	6,000	1/90,000	6,000 * 1/90,000 = 0.066 seconds

and so on.

What was all that about frame predictions?

How do you capture similarities between frames? From our earlier example, there is still huge savings to be had there - around 10X over just compressing each frame by itself!

Codecs handle this by splitting each frame into regions, and comparing differences between regions in adjacent frames. Two popular modern codecs are H.264 and H.265. H.264 uses sections called macroblocks, and H.265 uses something called coding tree units. These work like macroblocks but with more sizes available.

So now that we calculated block differences for 2 consecutive frames, we can store the full first frame, and only the delta for the second.

That second frame is a P frame. Once the first frame is decoded, the second frame can be decoded by applying those changes to the first frame.

B frames work the same way, except using differences between the frame before and after. Hence bidirectional.

Remember when we said you need a powerful processor to decode frames fast enough? This is why. In addition to PTS, frames also have something called DTS - decode timestamp. This indicates to the decoder when frames should be decoded. Now your timestamps might look something like this:

Frame	Type	DTS	PTS
1	I frame	0 (first decoded)	3,000
2	P frame	1,000 (second decoded)	6,000
3	B frame	7,000 (fourth decoded)	9,000
4	I frame	5,000 (third decoded)	12,000

The third frame, a B frame, must wait until frames 2 and 4 are decoded.

Keyframes are the fastest to decode - they don't rely on any other frames. P frames rely on the frame before them, but that's not really an issue. It had to be displayed earlier, so it has necessarily already been decoded. B frames are another animal. They rely on later frames being decoded, before they can be decoded themselves.

The window between DTS and PTS is usually short. Decoded frames take up a lot of memory, so you don't want to have more sitting around than absolutely necessary. Doubling+ the work you need to display a frame on time can make for tight deadlines.

B frames have the highest decoding performance requirements, but also generally the most compression. You can always compress less, and get faster decoding. But then you have much bigger filesizes.

So now we can store keyframes occasionally, when the whole screen needs an update. We can store P frames, storing only the differences between a frame and the last. And we can store B frames, storing differences between the frames both before and after, for maximum space savings. And that’s how you get a video file 0.4% as big as the original.

APSE Update

2019-04-18T12:00:00-05:00

I've been working on improving Apse since I initially released it two months ago. Version 1.4 is now available. There are some exciting new features!

Apse is self-aware. The QR-code-ish squares in the corners of the window are markers for Apse to locate itself. If Apse sees itself in a snapshot, it redacts that portion of the screen so you don't get Apse searches appearing in Apse results, appearing in Apse searches, ad infinitum.
Smart image processing: Apse auto-detects dark backgrounds to drastically improve accuracy in e.g. dark-theme windows and text editors.
Take Snapshot Now - you can manually trigger a snapshot at any time, in the app or with the Firefox extension.
Results are now sortable by relevance or date.

These features were both a lot of fun to work on and have been really useful for me. The smart image processing in particular was quite technically challenging, and I am very pleased with the results.

The changes for each version of Apse here. You can sign up for a trial or upgrade your license here!

FAA Part 107 Waiver Search

2019-03-01T12:00:00-06:00

The FAA allows pilots to fly drones for commercial operations if they obey part 107 rules. They also pilots who meet sufficient safety requirements to operate outside the normal regulations via a waiver.

I had been compiling information on the number of part 107 pilots, and thought it would be useful to find pilots with waivers nearby.

The FAA publishes that information in a poor format for searching, so I had the idea to make an interface to find pilots with a particular waiver, or near a particular location. It's a geospatial-enabled waiver search.

It is live at waivers.foxrow.com. If you're interested in using it let me know!

APSE - A Personal Search Engine

2019-02-16T12:00:00-06:00

APSE is an application I created to help me find things on my computer. It stands for A Personal Search Engine.

What it is

Apse is a desktop application that works as a search engine for anything that you see on your computer. As you use your computer, it captures text that appears anywhere on your screen - in a spreadsheet, text doc, web page, PDF, you name it. It even captures text inside videos and images. Everything is stored for you to search later.

How it works

Apse runs in the background of your desktop or laptop. You use your computer normally, and Apse archives your history in its search index.

Apse periodically takes a screenshot of your monitor and extracts any text. That text is stored in a search index, so you can search it like any other search engine.

Apse automatically suggests spelling corrections - did you mean X? Apse also lets you search forward or backward in time, if you know you are close but one hit doesn't have what you are looking for.

When you find what you're looking for, Apse shows you what was on your screen at the time, along with all the text it extracted, and the timestamp:

Privacy

Apse doesn't run in the cloud at all - it all stays local on your machine. Screenshots of your computer are really sensitive information, so everything Apse records stays on your computer, where it belongs.

If you want Apse to stop recording temporarily for whatever reason, you can pause and unpause it at any time. You can also delete a hit or range of hits if something gets recorded you would rather not have saved on your hard drive.

Get Apse

You can find out about Apse at apse.io. The latest version adds a feature where Apse can recognize itself in screenshots, and it will disregard that region of the image so as to not pollute your history with the things you are searching for.

You can sign up here. I hope you find it as useful as I do!

Using reCAPTCHA v3 with Django

2018-12-12T12:00:00-06:00

reCAPTCHA is a system to prevent spam and abuse of web services. It's easy to integrate with Django. There are three steps to using v3 reCAPTCHA in your web app:

Create the reCAPTCHA application
Integrate with your frontend
Validate the response from Google

Create the reCAPTCHA application

First, register a new reCAPTCHA application at https://www.google.com/recaptcha/admin:

Copy the value in the Secret Key box. Put this in your Django settings:

GOOGLE_RECAPTCHA_SECRET_KEY = <secret_key>

You probably don't want to hardcode the value in your settings.py. You can store it in a separate config file, and read it in via configparser.

Integrate with your frontend

In the "Client side integration" section of the reCAPTCHA admin page, copy the first snippet:

<script src='https://www.google.com/recaptcha/api.js?render=<YOUR_SITE_KEY>'></script>

Copy the one from your admin page, not the snippet above. Or replace <YOUR_SITE_KEY> with your actual site key. Put that tag in your template's <head> tag.

Copy the second snippet from the reCAPTCHA admin page:

<script>
    grecaptcha.ready(function() {
        grecaptcha.execute('<YOUR_SITE_KEY>', {action: '<ACTION_NAME>'})
        .then(function(token) {
            <RESPONSE_ACTION>
        });
    });
</script>

Put that in the <body> tag of the page you want to use reCAPTCHA on. You'll need to replace <YOUR_SITE_KEY> with your actual site key. The version on your reCAPTCHA admin page should already have it. You also need to replace <ACTION_NAME> with an appropriate value - if you are using it to validate user signups, you could use signup for example.

The next step is to replace <RESPONSE_ACTION> with some javascript. If you are using reCAPTCHA to protect form submissions, you might have this form on the same page:

<form method="post" id="form">
    <input type="email" name="email">
    <button type="submit">Sign up</button>
    </div>
</form>

Google will return a token to the client. That token needs to be passed to your server for validation. You can do that like this:

<script>
    grecaptcha.ready(function() {
        grecaptcha.execute('<YOUR_SITE_KEY>', {action: '<ACTION_NAME>'})
        .then(function(token) {
            document.getElementById("form").appendChild(document.CreateElement(`<input type="hidden" name="g-recaptcha-response" value=${token}`);
        });
    });
</script>

The form will now contain an extra item named g-recaptcha-response on submission.

Validate the response from Google

In the view that processes your form, you need to separately verify the submitted token is valid. If you are using a class-based view:

def form_valid(self, form):

    # get the token submitted in the form
    recaptcha_response = self.request.POST.get('g-recaptcha-response')
    url = 'https://www.google.com/recaptcha/api/siteverify'
    payload = {
        'secret': settings.GOOGLE_RECAPTCHA_SECRET_KEY,
        'response': recaptcha_response
    }
    data = urllib.parse.urlencode(payload).encode()
    req = urllib.request.Request(url, data=data)

    # verify the token submitted with the form is valid
    response = urllib.request.urlopen(req)
    result = json.loads(response.read().decode())

    # result will be a dict containing 'success' and 'action'.
    # it is important to verify both

    if (not result['success']) or (not result['action'] == 'signup'):  # make sure action matches the one from your template
        messages.error(self.request, 'Invalid reCAPTCHA. Please try again.')
        return super().form_invalid(form)

    <finish your form processing...>

That's it! Now you have integrated reCAPTCHA with your Django app.

This post was written using Python 3.7.0 and Django 2.1.4.

Drone Pilots November 2018

2018-11-05T12:00:00-06:00

Here are some statistics on drone pilot activity in the US. I obtained all the data from publicly available sources on faa.gov. The data is current as of November 1, 2018. If this is interesting or you want more information, get in touch.

Remote Pilot Certifications

As of November 1 2018, there are at least 70,882 pilots with a part 107 UAS (drone) certificate. Following the 60% rule, that gives 118,136 112,700, an increase of 5,400 in the last month.

Part 107 Waivers

As of today, there have been 2,201 waivers granted to 2,020 individuals at 1,732 companies. That represents an increase of 52 waivers since last post.

The first waiver to expire does so November of this year. Breakdown of waivers granted by type:

Regulation	Type	Count
107.29	Daylight operation	2023
107.41	Operation in controlled airspace	106
107.35	Operation of multiple small unmanned aircraft	40
107.31	Visual line of sight aircraft operation	25
107.51(b)	Altitude	18
107.39	Operation over human beings	15
107.33(b)	Visual observer	15
107.51(d)	Minimum distance from clouds	10
107.51(c)	Minimum visbility	10
107.33(c)(2)	Visual observer	9
107.25(b)	Operation from a moving vehicle or aircraft	4
107.39(a)	Operation over human beings (part of operation)	2
107.51(a)	Maximum groundspeed	1
107.39(b)	Operation over human beings (covered)	1
107.33(c)	Visual observer	1

The number of total waivers granted over time:

Drone Pilots October 2018

2018-10-16T12:00:00-05:00

Here are some statistics on drone pilot activity in the US. I obtained all the data from publicly available sources on faa.gov. The data is current as of October 1, 2018. If this is interesting or you want more information, get in touch.

Remote Pilot Certifications

As of October 1 2018, there are at least 67,631 pilots with a part 107 UAS (drone) certificate. Following the 60% rule, that gives 112,700, an increase of 4,700 in the last month.

Part 107 Waivers

As of today, there have been 2,149 waivers granted to 1,977 individuals at 1,685 companies. That represents an increase of 129 waivers since last post.

The first waiver to expire does so November of this year. Breakdown of waivers granted by type:

Regulation	Type	Count
107.29	Daylight operation	1975
107.41	Operation in controlled airspace	106
107.35	Operation of multiple small unmanned aircraft	38
107.31	Visual line of sight aircraft operation	24
107.51(b)	Altitude	17
107.39	Operation over human beings	14
107.33(b)	Visual observer	14
107.51(d)	Minimum distance from clouds	10
107.51(c)	Minimum visbility	10
107.33(c)(2)	Visual observer	8
107.25(b)	Operation from a moving vehicle or aircraft	4
107.51(a)	Maximum groundspeed	1
107.39(b)	Operation over human beings (covered)	1
107.39(a)	Operation over human beings (part of operation)	1
107.33(c)	Visual observer	1

The number of total waivers granted over time:

BOGO - An Image Compression Format

2018-09-23T12:00:00-05:00

BOGO (Basically Optimal Graphics Ontology) is a lossless image compression format. BOGO derives its name from bogosort. Unlike traditional computationally-wasteful image formats that let your underutilized CPU sit around idle, BOGO maximizes your computer's time spent doing what it loves: computing.

Contrary to rumor, BOGO does not stand for Blockchain-Oriented Graphics Orifice. Why would you use a blockchain to store an image? That would be silly.

A proof-of-concept encoder and decoder are included in the reference implementation.

Usage:

# encoding
$ python encode.py input.jpg output.bogo

# decoding
$ python decode.py input.bogo output.jpg

BOGO image spec v0.0.1

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

BOGO files should use the file extension ".bogo".

BOGO files must adhere to the following format:

Byte #	Content
0-3	Magic number containing the ASCII for "BOGO" (Decimal `66` `79` `71` `79`)
4	Width of the image in pixels. Trust me, you don't want an image larger than 256x256
5	Height of the image in pixels
6	Image mode. 0 corresponds to black and white, 1 to grayscale, and 2 to RGB
7-22	Image checksum
23-N	Image data

All numerical values must be unsigned integers in big-endian format.

Image mode

BOGO supports 3 image modes: black and white, grayscale, and RGB.

For black and white images, each pixel value is stored as 1 or 0.

For grayscale images, each pixel is stored as 0-255, with 8 bits per pixel.

For RGB images, each pixel is stored as a 3-tuple of 0-255 values. Red must be the 0th value in the tuple, followed by green, followed by blue.

Checksum

A BOGO header contains a checksum to verify that the image has been correctly reconstructed. To compute the checksum of a given image, concatenate each pixel value in raster order and compute the MD5 of the result. For multi-channel images, data shall be concatenated per-pixel rather than per-channel. That is, concatenate the RGB values of the 0th pixel, followed by the RGB values of the 1st pixel, etc.

Image data

To compress image data, BOGO pixel values are stored using run-length encoding. To maximize compression, pixel values must be sorted before encoding. An encoded pixel is stored using two bytes for the count, followed by the pixel value. For simplicity, black and white pixel values are stored using a full byte, rather than a single bit.

For black/white and grayscale images, this gives encoded pixels a width of 3 bytes. For RGB images, 2 bytes for count plus 3 bytes for value gives an encoded pixel width of 5 bytes.

Decoding

Reconstruct the list of pixel values by decompressing the run-length encoded image data.
Randomly shuffle the pixel values.
Compute the MD5 checksum.
If it matches the checksum stored in the file header, you are done. If not, go to step 2.

The BOGO reference implementation is available on Github.

Howe Ridge Fire

2018-09-18T00:00:00-05:00

I visited Glacier National Park the second week of August this summer. We stayed on the south end of Lake McDonald. One morning after a couple days (and a thunderstorm the previous night), we noticed smoke rising from behind a ridge a few miles across the lake.

It turned out lightning had ignited dry timber, and started a forest fire. We saw several "Super Scoopers" and helicopters getting water from the lake to slow the fire's spread. We watched the fire grow throughout the first day and night. Farther north along the lake, all people were evacuated the first night - those are the lights you can see driving down the road.

We didn't need to evacuate, though the town we were staying in was under an evacuation warning. The fire continued to grow through the time we left. As of this writing, the fire has grown to 14,000+ acres.

Are Punters Forever?

2018-09-04T12:00:00-05:00

At 538, Benjamin Morris wrote kickers are forever. He presents a compelling case that kickers of all ages have been improving steadily at all distances since the 1960's.

I wanted to see if punters have done the same. Using play-by-play data from 2009 to 2017, I calculated how much of the available yardage punters are eating up. An ideal punt goes out of bounds at the 1 yard line, with no chance for a return. Given that target outcome, I defined punting fraction as the percentage of available yards a punt gains for the kicking team. A kick from the 50 to the opponent's 1 gives 49 / 50, or 0.98. Notice you can have a negative punting fraction after a long return.

Yardage

Here's what the punting fraction looks like for every year since 2009. It seems that punters are getting better - there is a modest upward trend.

Broken down by distance, it's slightly more dramatic. At virtually all ranges, the punting fraction has improved.

40-50 yards away is pretty noisy, but still the recent years come out on top.

Touchbacks

A touchback is a pretty bad outcome for a punter - you would rather have it come up 5 yards short instead of give up another 15. Here's the touchback rate by year:

Again, a modest decrease across the board. The breakdown by distance shows similar improvement:

It's a noisy signal, especially once your on the short side of the 50, but the more recent years fall to the bottom of the pile.

Returns

In an ideal world, the returner fair catches the kick, or the coverage team downs the ball with no chance for a return. What does the return rate progression look like?

I'm not sure what caused the uptick in 2015 - I don't think there were any rules changes relevant to punt returns. The breakdown on return rate by distance:

Same story, different chart. Slight improvements, at most distances. Return yardage follows an interesting path:

I'm not sure what caused the uptick in 2010 and 2011. The rules changes don't seem like they would affect punt returns dramatically. Since the high in 2011, return yardage is down almost 20%.

In the end, we see modest improvement across nearly all fronts - more yardage, less touchbacks, fewer and shorter returns.

I wasn't able to find any stats on hang time, I imagine that would be interesting to chart as well.

While their improvement may not be as dramatic as their kicking brethren, it looks like punters are forever too.

Here's every punt from 2009-2017. There are a lot of data points, but a couple features jump out - the touchback line, a few penalty lines as well. Also interesting that TD returns are a danger at pretty much any distance. That makes sense - once the returner has beaten all 11 would-be tacklers, they can usually run as long as they need to.

Drone Pilots September 2018

2018-09-04T12:00:00-05:00

Here are some statistics on drone pilot activity in the US. I obtained all the data from publicly available sources on faa.gov. The data is current as of September 1, 2018. If this is interesting or you want more information, get in touch.

Remote Pilot Certifications

As of September 1 2018, there are at least 64,867 pilots with a part 107 UAS (drone) certificate. Following the 60% rule from last month, that gives 108,000, an increase of 8,000 in the last month.

Part 107 Waivers

As of today, there have been 2,020 waivers granted to 1,855 individuals at 1,565 companies. That represents an increase of 21 waivers since last post.

The first waiver to expire does so November of this year. Breakdown of waivers granted by type:

Regulation	Type	Count
107.29	Daylight operation	1850
107.41	Operation in controlled airspace	106
107.35	Operation of multiple small unmanned aircraft	37
107.31	Visual line of sight aircraft operation	22
107.51(b)	Altitude	17
107.39	Operation over human beings	12
107.33(b)	Visual observer	12
107.51(d)	Minimum distance from clouds	10
107.51(c)	Minimum visibility	10
107.33(c)(2)	Visual observer	6
107.25(b)	Operation from a moving vehicle or aircraft	4
107.51(a)	Maximum groundspeed	1
107.39(a)	Operation over human beings	1
107.33(c)	Visual observer	1

The number of total waivers granted over time:

Graph Perturbation

2018-08-29T12:00:00-05:00

Tom7 has an interesting video about portmanteaus. I took the idea to make a program to draw a graph as you move around nodes to shorten the lines (called edges) between them. Here's what a small run looks like:

Source on github here.

Drone Pilots August 2018

2018-08-28T12:00:00-05:00

Here are some statistics on drone pilot activity in the US. I obtained all the data from publicly available sources on faa.gov. If this is interesting or you want more information, get in touch.

Remote Pilot Certifications

As of August 28 2018, there are at least 61,613 pilots with a part 107 UAS (drone) certificate. I say at least because the FAA allows pilots to exclude themselves from the public database, in which case they do not show up in the data set.

Per this press release, the FAA hit 100,000 certificates about a month ago, so a first order estimate puts the number of pilots who don't exclude themselves around 60%.

The FAA publishes new data every month, so in a couple days there will be more available. This is the first month I have obtained data, so I don't have anything to compare to yet.

Part 107 Waivers

As of today, there have been 1,999 waivers granted to 1,854 individuals at 1,544 companies.

The first waiver to expire does so November of this year. Breakdown of waivers granted by type:

Regulation	Type	Count
107.29	Daylight operation	1829
107.41	Operation in controlled airspace	106
107.35	Operation of multiple small unmanned aircraft	37
107.31	Visual line of sight aircraft operation	22
107.51(b)	Altitude	17
107.39	Operation over human beings	12
107.33(b)	Visual observer	12
107.51(d)	Minimum distance from clouds	10
107.51(c)	Minimum visibility	10
107.33(c)(2)	Visual observer	6
107.25(b)	Operation from a moving vehicle or aircraft	4
107.51(a)	Maximum groundspeed	1
107.39(a)	Operation over human beings	1
107.33(c)	Visual observer	1

The number of total waivers granted over time:

Technical Writing

2018-07-30T00:00:00-05:00

Years ago I remember coming across a comment on edwardtufte.com about technical writing. I had forgotten exactly what it said, but remembered it was short, had a snappy title, and followed its own advice.

I discovered it again. The page at the original url is long gone, but the Wayback Machine comes to the rescue:

Best Writing Advice for Engineers I've Ever Seen. Period.

How to make engineers write concisely with sentences? By combining journalism with the technical report format. In a newspaper article, the paragraphs are ordered by importance, so that the reader can stop reading the article at whatever point they lose interest, knowing that the part they have read was more important than the part left unread.

State your message in one sentence. That is your title. Write one paragraph justifying the message. That is your abstract. Circle each phrase in the abstract that needs clarification or more context. Write a paragraph or two for each such phrase. That is the body of your report. Identify each sentence in the body that needs clarification and write a paragraph or two in the appendix. Include your contact information for readers who require further detail.

– William A. Wood, September 8, 2005

More accurately, the Wayback Machine has an archive of a page on Jottit, which appears to be a pastebin-style site. It looks like Jottit is running on Heroku, and currently returns an error for any page. Shame.

Here is the original link, the original jottit, and the archived version on the Wayback Machine. Thanks William Wood, Edward Tufte, and archive.org!

Milwaukee Air Show

2018-07-29T00:00:00-05:00

Update July 2022: Added photos from the 2022 show, featuring the Blue Angels and an F-22.

The 2018 Milwaukee air and water show featured the USAF Thunderbirds. I got some photos of their practice flight around downtown Milwaukee the day before the show. Super loud!

Troubleshooting Raspberry Pi Headless Wifi Config

2018-05-31T12:00:00-05:00

There are several models of Raspberry Pi with built-in wireless support. They can be configured to connect to a wireless network before powering on, by creating a file named wpa_supplicant.conf in the boot partition of your SD card.

If you have your card properly configured, but the Pi won't connect, make sure you have UNIX-style \n line endings and not windows-style \r\n endings in wpa_supplicant.conf.

This is what the file should look like:

country=us
update_config=1
ctrl_interface=/var/run/wpa_supplicant

network={
 scan_ssid=1
 ssid="My Network Name"
 psk="your password"
}

Make sure you have the right line endings, place that file in the boot partition, and you're good to go.

Markdown Tables in Sphinx

2018-05-30T12:00:00-05:00

Sphinx is a really useful tool, but one shortcoming is its markdown support. Specifically, it uses the recommonmark specification, which does not support tables. I wrote a Sphinx extension to render tables authored in markdown.

It can be installed via pip install sphinx-markdown-tables.

If you don't already have Sphinx configured to parse markdown, run pip install recommonmark, and edit conf.py to tell Sphinx to parse your markdown files:

source_parsers = {
    '.md': 'recommonmark.parser.CommonMarkParser',
}

source_suffix = ['.rst', '.md']

Once Sphinx is configured appropriately, add sphinx-markdown-tables to extensions, like so:

extensions = [
    'sphinx-markdown-tables',
]

You can find the source on github here.

Cyphercon 2018

2018-05-09T12:00:00-05:00

I gave a talk on Steganography at Cyphercon 3.0. I want to give thanks to the organizers who put everything together. It was a blast - really fun conference with a lot of things to see, cool demos, and gadgets to try out.

The slides from my talk are available here. There was video recorded of the talk, I will update this when it becomes available.

Tips for drone video - camera settings

2018-01-09T12:00:00-06:00

Drones can get great aerial photos and video, but they're only as good as the person piloting them. In the right hands, a phone camera can get fantastic shots, and all the equipment in the world won't help if you don't know how to use it.

The first time you take your drone out, you're probably not getting dramatic, hollywood shots.

Here's some tips I've found help me get better photos and videos.

The biggest thing is that aerial photos and video are photos and video - all the same rules and tips apply, your camera just happens to be up in the air.
First things first, find the image quality settings and turn the image quality all the way up. If your drone supports RAW, go ahead if you want to use it. For most purposes, the highest-quality JPG will be more than sufficient. Memory cards give you effectively infinite space - no sense saving room for photos you'll never take.
Another setting you should turn on is the grid. This lets you easily check your composition against the rule of thirds. It still makes for good shots when you're airborne!
Last thing in the settings - white balance and color settings. Most drones have pretty blah defaults. If you're flying outside, you probably want sunny or cloudy white balance. If your drone supports it, you can enable different color modes for a more vibrant look. DJI has a few options, try them out and see which you like best. D-Cinelike in particular works pretty well.

I plan on making more tips for aerial video. If you're interested in more, check out Drone Stable and sign up for the newsletter. Good luck flying :)

What I learned in 2017

2018-01-05T12:00:00-06:00

Here are some things I learned in 2017, a summary of what I worked on the last year, and what I hope to accomplish in 2018.

A lot of them are things that have been said before. I think they bear repeating partially thanks to the lucky 10,000 effect. Another reason is what's obvious to you isn't necessarily so to others - the most heavily-trafficked page on the site by far was Installing OpenCV in a virtualenv, hardly groundbreaking research.

I include my future self in this: the page I visited the most in 2017 was generating Django secret keys. I wrote it, and I still refer to it all the time. You in the future is a different person.

Consulting

2017 was a big year for drones. I consulted primarily on the data processing side. I got to work on some really fun projects.

Many of the interesting things I helped build were photo-based: panoramas, orthomosaics (fancy name for maps), image recognition. Others used photos for more complex products: photogrammetry, point clouds, 3d modeling, video processing, etc.

A big highlight was building out an automated aerial mapping data pipeline. I worked with a client collecting drone photos and video from sites across the country. We were able to consolidate their process and automate much of it. We got a ~5x improvement in throughput over manually processing the data.

Speaking

I gave my first conference talk at PyOhio and reprised the talk at Milwaukee Code Camp. It was a blast.

I also gave my first, second, and third presentations at a meetup. These were fun too, and I made some friends working on really interesting stuff as well.

Personal projects

I created DIL, software for processing drone imagery.

I also created Drone Stable. I started showing it to pilots in October, and I've gotten a really good response. Here's what it does:

Visit dronestable.com to check it out.

Goals for 2018

I have not made as much progress on the computer vision cheat sheet as I would like. I want to write at least one item per month for it, either adding an example to the list or creating new items.
Continue consulting with drone companies. I think businesses can get a lot of value out of machine learning and automation. If you're one of those people and want to talk, email me!
Keep up on public speaking - I would like to give a talk at two conferences this year.
I am thrilled about the feedback I've gotten on Drone Stable. I want to dedicate the time it needs to really grow in 2018. It will likely be my biggest project for a while. There's certainly enough shaky footage around. I hope it can start to pay the rent :)
Publish the 2017 year in review on time.

See you in 2018!

DIL - Drone Imaging Language

2017-12-13T12:00:00-06:00

I work a lot with drone data. Two common tasks are stitching 360 degee panoramas and detecting objects in an image.

Both of those are technically complicated - panorama stitching involves feature point detection, correlation, determining homographies, seam carving, and more.

Object detection is a whole field of research unto itself, but suffice to say it is not a simple process.

There are programming tools for both of those tasks, but if you're not a programmer it is going to be difficult to take advantage of most of them.

One way to deal with that barrier to entry is to create a DSL, or domain specific language.

The idea is not to create a general-purpose language, but one tailored to a specific use case (domain). This lets you create a simple interface, for example one that closely follows English, so non-technical users can take advantage of it.

To that end, I created a DSL for drone imagery: DIL, the Drone Imaging Language.

It's deliberately designed to be very simple. Here's a sample of DIL code:

load DJI_001.jpg DJI_002.jpg DJI_003.jpg
highlight boat person
save

That will load the 3 specified images, highlight any boats or people in them, and save the results. There's 80 object classes available for highlighting, see classes.txt for the full list.

There are 5 commands available in DIL:

load
highlight
stitch
show
save

Commands are run one at a time from top to bottom. Each command does the following:

load - Load one or more images. Loaded images are the ones that will be stitched and/or highlighted. This should be the first command in a DIL file. There may be more loads in a file. The most load will replace old ones - save your operations if you don't want to lose them.

highlight - Highlight the given classes in the loaded images. More than one class may be specified. This is what a highlighted image looks like:

stitch - stitch the loaded files into a panorama. This may be a regular pano or 360.

show - launch a web browser to visualize a panorama. Requires a loaded set of files to have already been stitched. This is what the web viewer looks like:

save - save changes to files. This includes any highlighted images or stitched panoramas.

I talked about DIL at the end of my talk about computer vision and image processing. There is video here if you want to learn more. Happy programming!

What is Bootstrap?

2017-12-13T12:00:00-06:00

Maybe you're heard of Bootstrap, but don't know exactly what it is, or how to use it? The landing page description is less than helpful:

Build responsive, mobile-first projects on the web with the world's most popular front-end component library. Bootstrap is an open source toolkit for developing with HTML, CSS, and JS. Quickly prototype your ideas or build your entire app with our Sass variables and mixins, responsive grid system, extensive prebuilt components, and powerful plugins built on jQuery.

Here's a quick primer with what you need to get started.

What is Bootstrap?

Twitter Bootstrap? It's a UI framework for websites.

Ok, but what does that mean?

Web pages are a combination of HTML, CSS, and JavaScript. Bootstrap is a set of CSS and (optional) JavaScript files with a boatload of common widgets and styles for making a page look good with a minimum of time and effort.

Why would I use it?

Unstyled web pages look pretty rough, especially on very large or small screens. Bootstrap gives you tools for easily styling a web page. The bigggest win in my opinion is ease of use - add a CSS class to your tags and you're off to the races.

How do I get it?

They're just regular old CSS and JS files. You can either link them from the Bootstrap CDN:

<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.3/umd/popper.min.js" integrity="sha384-vFJXuSJphROIrBnz7yo7oB41mKfc8JzQZiCq4NCceLEaO4IHwicKwpJf9c9IpFgh" crossorigin="anonymous"></script>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/css/bootstrap.min.css" integrity="sha384-PsH8R72JQ3SOdhVi3uxftmaW6Vc51MKb0q5P2rRUpPvrszuE4W1povHYgTpBfshb" crossorigin="anonymous">
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/js/bootstrap.min.js" integrity="sha384-alpBpkh1PFOepccYVYDB4do5UnbKysX5WZXm3XxPqe5iKTfUKjNkCk9SaVuEZflJ" crossorigin="anonymous"></script>

Or you can download the files and serve them yourself.

How do I use it?

Once the files are included in your HTML, you get Bootstrap components by applying CSS classes to HTML tags. Mostly divs. Once you have the appropriate files linked in your HTML document, add the classes you are interested in. For instance:

<button class="btn btn-primary">CLICK HERE</button>

Renders like this:
There's a ton more components here, and a good starter template here.

You probably want to use the skeleton, at least in the beginning. It is responsive by default, and includes all the files you need.

Ok that's easy enough, but what about page layout?

Glad you asked! Bootstrap also includes a nifty grid system. The general pattern is this:

<div class="container"> <!-- wraps all your content -->
    <div class="row"> <!-- wraps a single row -->
        <div class="col"> <!-- wraps a single column -->
            <p>My content in column 1</p>
        </div>
        <div class="col">
            <p>My content in column 2</p>
        </div>
        <div class="col">
            <p>My content in column 3</p>
        </div>
    </div>
</div>

That will get you a centered, reasonably-sized container with 3 equally wide columns of text. You can specify widths, content alignment, and much more by adding more CSS classes. Tons more on the layout system here.

Great, anything else I should know?

The project documentation is very extensive. Find out more about the components, grid system, and utilities here.

Happy styling!

Django at Milwaukee Python

2017-10-11T12:00:00-05:00

On Wednesday night, the Milwaukee Python group met up to discuss Django. The meeting was hosted by Digital Measures in Milwaukee's third ward. We had a great turnout and lots of good questions.

Andrew Glassman and I both spoke on Django, its features, and how to use it. The notes from my portion of the talk are here.

The Django project I mentioned is Fast Ergs. A less-complicated app using Flask lives at Needsmorelensflare.com

Thanks to Digital Measures and MKE Python for hosting us!

Detecting circles in an image with Hough transforms

2017-09-13T12:00:00-05:00

Part of a series of examples from the CV cheat sheet. Click here for other computer vision applications.

Detecting circles in an image is a common task in computer vision. One way to do this is with the Hough (pronounced "Huff") transform. The math behind it is beyond the scope of this example, but we should be able to get you off the ground using it with OpenCV and Python. We're going to assume you have OpenCV installed already and are able to use it in Python.

Take an input image, say, these solar system moons:

Let's detect the circles in this image. The gist is this:

Open an image
Detect circles in it
Draw those circles back on the image (you don't need to do this necessarily, but it's a nice visual)

Here's the code to extract circles in Python:

# import libraries we'll need
import cv2
import numpy as np

# read in your image
img = cv2.imread('moons.jpg')

# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# detect circles in the image
# dp and minDist are parameters that you can tune depending on
# your data.
# dp is the accumulator resolution. Think of it as accuracy of circle center location. Lower values take longer
# minDist is the minimum distance permitted between circle centers
# param1 is the upper threshold for the edge detection preprocessing. If you have low-contrast circles, increasing this may help
# param2 is 
# param1 and param2 are optional.
circles = cv2.HoughCircles(img, cv2.HOUGH_GRADIENT, dp=5, minDist=400, param1=350, param2=25)

# circles are returned in (x, y, radius) format
for x, y, radius in circles[0]:
    cv2.circle(img, (x, y), radius, (0, 255, 0), cv2.LINE_AA)

# write the image to disk
cv2.imwrite('houghcircles.jpg', img)

Here's the result:

As you can see, there is still some error! The most elliptical of our Galilean satellites wasn't detected at all, and the radius of a couple others is off. Tweaking the parameters for your use case will help this. If you want to count almost-circles or not count them, how sharp the edge needs to be to count as a detection, and so on.

PyOhio 2017

2017-07-30T12:00:00-05:00

I was really happy to give a talk at PyOhio this year. My presentation was To Infinity and Beyond, a talk about image processing, computer vision, and astronomy.

You can find the slides from my talk in PDF format here.

The programming language I talked about, DIL, lives on GitHub. Contributions are welcome!

Video of the talk:

I'd like to give a big thanks to the organizers of the conference for putting on such a great event, and for having me!

Slides under CC-BY 4.0.

Which commits to squash when rebasing in Git

2017-06-16T12:00:00-05:00

git rebase is a really powerful tool, but I always forget which commits to pick and squash. Suppose your commit history looks like this:

bd7bf7d this is the commit message I want to keep
2313327 checkpoint commit 2
d4ac9ee checkpoint commit 1
c56d9c3 latest commit on master (don't change this one)

You want to run git rebase -i HEAD~3, which will bring up your editor:

pick d4ac9ee checkpoint commit 1
pick 2313327 checkpoint commit 2
pick bd7bf7d this is the commit message I want to keep

# Rebase c56d9c3..bd7bf7d onto c56d9c3 (3 command(s))
#
# Commands:
# p, pick = use commit
# r, reword = use commit, but edit the commit message
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
# f, fixup = like "squash", but discard this commit's log message
# x, exec = run command (the rest of the line) using shell
# d, drop = remove commit
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#
# Note that empty commits are commented out

Change the second two picks to s or squash. The editor should look like this:

pick d4ac9ee checkpoint commit 1
squash 2313327 checkpoint commit 2
squash bd7bf7d this is the commit message I want to keep

# Rebase c56d9c3..bd7bf7d onto c56d9c3 (3 command(s))
#
# Commands:
# p, pick = use commit
# r, reword = use commit, but edit the commit message
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
# f, fixup = like "squash", but discard this commit's log message
# x, exec = run command (the rest of the line) using shell
# d, drop = remove commit
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#
# Note that empty commits are commented out

Save the file and exit. Now you need to fix the commit messages:

# This is a combination of 3 commits.
# The first commit's message is:
checkpoint commit 1

# This is the 2nd commit message:

checkpoint commit 2

# This is the 3rd commit message:

this is the commit message I want to keep
[...]

Delete everything up to "this is the commit message I want to keep". Save and quit your editor. Now the rebase will proceed. Your commit history should now look like

5151034 this is the commit message I want to keep
c56d9c3 latest commit on master

Note the latest commit has a new hash - it's a new commit, created out of the three previous commits, which are now gone. So use it carefully!

Happy coding!

Stitching 360 degree panoramas with Hugin

2017-06-15T12:00:00-05:00

In my last post, I talked about stitching panoramas with Photoshop. There is an open source application named Hugin that can do the same and more. It is dedicated panorama stitching software, so there are many more options available to you.

This guide uses Hugin 2016.2.0.

Hugin supports basic panorama stitching, the same as Photoshop. But where it really shines is 360 degree panoramas. With a panorama viewer like Pannellum, you can view panos much more easily on the web. Here's how you can create one:

Take a set of 360 degree panoramic photos. This is the same as a regular panorama, you just end up rotating 360 degrees. You can also stack multiple strips to get a bigger vertical field of view.
Open Hugin.
Go to Interface -> Expert
In the lower left quadrant of the window, click Load Images. Choose the images you want to stitch.
Under Feature Matching, Select Cpfind (multirow/stacked)
Click Create Control Points. This may take a few minutes.
Under Optimize, for Geometric, keep Positions (incremental, starting from anchor). Click Calculate. This should be fairly quick. Once it's done, click Yes.
For Photometric optimization, keep Low dynamic range. Click Yes.
Click on the Stitcher tab.
Leave projection as Equirectangular. This is the projection 360 degree panorama viewers are expecting.
If you want, you can change the canvas size. Clicking Calculate Optimal Size makes the panorama as big as if the original images were not resized at all. You probably don't want this to be bigger than 10,000 pixels wide.
If you like, you can click "Fit crop to images". This will trim any whitespace off the final panorama. This is useful if you have e.g. a hole in the sky directly above the camera.
Select the output format you want. It defaults to TIF, which can give very big image sizes. JPG is a nice alternative for smaller files.
Click Stitch! This may take a while, if the output size is large.
Your panorama is ready. While the raw image doesn't look like a panorama, your 360 pano viewer of choice will re-warp it back to its original state.

Happy shooting!

Stitching panoramas with Photoshop

2017-06-14T12:00:00-05:00

One of Photoshop's less frequently used features is the ability to stitch together photos into a panorama. Here's how you can go about making them.

This guide uses Photoshop CC 2017.1.1 on Windows 10.

In Photoshop, go to File -> Automate -> Photomerge.
The photomerge dialog will come up. You can experiment with different layouts, but I have found "Auto" does the best job.
Hit browse and select the files you want to stitch.
You can change the options selected at the bottom, but the defaults are a good starting point. They don't affect the stitching results, just the blending of images once they are aligned.
Hit Ok. Photoshop will load the images and start stitching. This may take a few minutes, depending on how big your final panorama is.

That's it. Once the stitching is complete, you can crop the image to your liking, make modifications, or anything else Photoshop permits.

Happy shooting!

Online QR code reader

2016-12-20T00:00:00-06:00

Choose an image to read a QR code:

Read QR code

QR-decoding code courtesy of LazarSoft, available under the Apache License.

Detecting lines in an image with Hough transforms

2016-12-02T12:00:00-06:00

Part of a series of examples from the CV cheat sheet. Click here for other computer vision applications.

Detecting straight lines in an image is a common task in computer vision. One way to do this is with the Hough (pronounced "Huff") transform. The math behind it is beyond the scope of this example, but we should be able to get you off the ground using it with OpenCV and Python. We're going to assume you have OpenCV installed already and are able to use it in Python.

Take an input image, say, this basketball court:

Credit: Wikimedia Commons

Let's detect the straight lines in this image. The gist is this:

Open an image
Detect edges in it
Use those edges to detect straight lines
Draw those lines back on the image (you don't need to do this necessarily, but it's a nice visual)

Here's the code to extract lines in Python:

# import libraries we'll need
import cv2
import numpy as np

# read in your image
img = cv2.imread('court.jpg')

# convert to grayscale so we can detect edges
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# use Canny edge detector to find edges in the image.  The thresholds determine how
# weak or strong an edge will be detected.  These can be tweaked.
lower_threshold = 50
upper_threshold = 150
edges = cv2.Canny(gray, lower_threshold, upper_threshold)

# detect lines in the image.  This is where the real work is done.  Higher threshold
# means a line needs to be stronger to be detected, so again, this can be tweaked.
threshold = 250
lines = cv2.HoughLines(edges, 1, np.pi / 180, threshold)

# convert each line to coordinates back in the original image
for line in lines:
    for rho, theta in line:
        a = np.cos(theta)
        b = np.sin(theta)
        x0 = a * rho
        y0 = b * rho
        x1 = int(x0 + 1000 * -b)
        y1 = int(y0 + 1000 * a)
        x2 = int(x0 - 1000 * -b)
        y2 = int(y0 - 1000 * a)

        # draw each line on the image
        cv2.line(img, (x1, y1), (x2, y2), (0, 0, 255), 1)

# write the image to disk
cv2.imwrite('houghlines.jpg', img)

Here's the result: We can see it found the majority of lines on the court - and quite a few in the skyline. This is why you'll want to tune the threshold in cv2.HoughLines(edges, 1, np.pi / 180, threshold) based on your needs to prevent false positives or negatives.

Computer vision cheat sheet

2016-11-15T12:00:00-06:00

Computer vision is a broad field - there are many algorithms and it can be hard to tell what's the best way to attack a given problem. This aims to provide a guide on what to use.

What this isn't

The bleeding edge. CV moves very quickly. I haven't always heard about the newest bestest way to analyze $YOUR_FAVORITE_THING. Email me and I'll take it under consideration though!
The best way. "Best" often isn't even well-defined. Preprocessing time, CPU usage, memory usage, accuracy, and simplicity are all axes you might value differently for different applications. All else equal, I will try and show the simplest way to implement a solution.
Comprehensive. There are many, many, areas in CV. Covering them all is beyond the scope of this guide.

What this is

An illustration of what is possible. It's not always obvious a task can be done, let alone knowing what magic words to Google. I want to provide an overview of what is out there.
Translation. Often you know what you want, but no way to know that it's called, say, histogram backprojection. This should help bridge that gap.
A living document. I'll be adding entries, updating, and adding examples. If you have suggestions for more, let me know!

I want to...	You should use...
Detect lines in an image	Hough Transform
Detect circles in an image	Circular Hough Transform
Find images similar to the one I have	phash, dhash
Find images containing a particular object	Feature detector - orb, sift, surf, kaze/akaze
Find images containing any particular object	Haar cascade, convolutional neural net
Track an object in a video	Meanshift, camshift
Detect faces in an image	Haar cascade
Detect people in an image	Histogram of Oriented Gradients (HOG) people detector
Remove defects (like passers-by) from an image	Median filter
Detect motion in an image	Background subtraction + frame differences
Read words or numbers in an image	Optical character recognition (OCR)
Classify what an image contains (e.g. horse, Golden Gate Bridge, human, etc)	Deep learning, especially Convolutional Neural Nets

How to generate Django secret keys

2016-07-05T12:00:00-05:00

You accidentally committed your secret key to version control, or the server got compromised, or you lost control of the secret key for some reason. Using an online service to generate one is a bad idea - nobody else should ever have your secret key. How do you generate a new secret key? The otherwise-excellent Django docs are silent on this. Here's how Django generates one when you run startproject:

>>> import random
>>> ''.join(random.SystemRandom().choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50))

Generate a new one and you're good to go.

retread.py - detect duplicate frames in Video, TV, or Film

2016-06-05T12:00:00-05:00

The Force Awakens has a lot of shots that are callbacks to A New Hope - they evoke very particular scenes and shots, if not outright identical. That got me thinking about recycling shots in films. How could that be quantified?

A common task in image processing is finding sets of similar images. Often you have a picture, and you want to search for identical or near-identical copies from some large set. A few ways to do this are described here - pHash and dHash in particular are quite effective.

We can use this to our advantage. Given a video clip, we should be able to go frame-by-frame and hash each image. We can then count the number of frames that hash to the same value, telling us how many occurred earlier. I wrote up a python tool to do this: retread.py.

Retread measures how much any given frame is reused in a video clip. It can be a TV show, online video, any video file you can get your hands on. It spits out nice JSON, so you can plot it nicely via d3.

Now, let's see what some movies look like. First up is Mad Max: Fury Road, a 2015 action blockbuster with many fast cuts and short shots overall.

Often the most common frame ends up being black. This is usually credits, maybe a handful of frames from the beginning of the movie, and any fade-to-blacks in the middle. This shows up as big, solid bars near the beginning and end. The interesting stuff is in the middle. Here we see Max doesn't have any monster repetitive sections:

On the other hand, Memento, a film with a nonlinear story, has heavier and more spikes. The film is presented in sort of an outside-in fashion - the end comes first, then the beginning, then back to the end, working toward the middle the whole time:

Inception somewhat famously wraps multiple stories within one another, with the narrative jumping between all 3 in parallel. Christopher Nolan directed both Memento and Inception. He seems to be a fan of cutting across time and space. This results in a moderately busy graph:

Paprika, one of the inspirations for Inception, has even more. Flashbacks occur throughout the film, and it shows. The chart is almost totally full:

See retread here to analyze your own films, TV shows, or clips.

Online stroke watch

2016-05-15T12:00:00-05:00

Time

00:00.0

Rate

Start/Stop

Stroke

Reset

How to use: Hit Start/Stop to begin or pause the timer. At either the catch or the finish, hit 'Stroke'. You don't need the stopwatch running to calcuate rate. No downloads or app required!

If this was helpful to you, check out Virtual Cox. It's an online service for rowing video analysis and athlete tracking. I wrote some more about it here.

Introducing Virtual Cox

2016-04-15T12:00:00-05:00

Update: I'm changing the name of Virtual Cox to Fast Ergs. It now lives at fastergs.com. I'm using the name Virtual Cox for another project, coming spring 2017. You can subscribe for updates and to be notified when we launch here.

I'm pleased to announce the launch of a project I have been working on for a while. It's called Virtual Cox, and it's an online tool to help rowers and rowing coaches row faster and more efficiently. The homepage can be found at virtualcox.com.

In a sentence: Virtual Cox lets rowers and rowing coaches analyze video to go faster on the water.

Elevator pitch: Virtual Cox lets rowers and rowing coaches analyze video to go faster on the water. Video can be captured via any device, and data analysis tools enable closer critique of form, technique, and consistency. Sharing tools let coaches provide feedback to athletes for later review.

Even a short amount of footage can make rough edges glaringly obvious, even things the rowers don't realize they're doing themselves! Despite being such a rich source of feedback to both athletes and coaches, there are often barriers to using it effectively (or at all). Like photography, the best camera is the one you have with you. My goal is to give crews the tools to take and analyze more video.

Video can be uploaded from any device, so you don't have to bring along a video camera or mount any equipment on the boat. (Though you certainly can if you want!) Simply using your phone in the launch, GoPro up close, or video camera can take all the footage you need. Erging video works too, for those winter training sessions.

Once you've uploaded your video, tools allow you to slow down playback to closely review technique. For more advanced measurements, coaches and rowers can measure ratio, timing, and other metrics.

Coaches can save and easily share video and data with other coaches, coxswains, or athletes via a sharing link.

I'm always looking to hear from you! Send me an email at ryan@virtualcox.com, I read all of them, and try to get back to everyone. If it's been a few days and you haven't heard back from me, send me a quick ping. It's more likely it slipped off my radar than I took a personal disliking to your email. If you're a rower or coach looking to get faster on the water, sign up at Virtual Cox, and follow @virtualcox.

Brainfunction - a new programming language

2016-02-26T12:00:00-06:00

Some time ago, I wrote an interpreter for the language brainfuck. It's nice and all, but I thought, "you know what would really make this great? More inscrutable instructions!" How to extend it, though? It already has such a great feature set. You may remember it from hits such as:

Hilarious-looking source code!
Turing completeness!
It somehow manages ultra-verbosity in a language with only 8 valid characters.

Despite that saliva-inducing list, one item you can never get enough of in computing is recursion. So let's add functions! Brainfuck already has an array of data cells, let's expand that to an array of functions to call as well.

Official brainfunction v0.1 spec (subject to revision):

The official brainfunction repository can be found at https://github.com/ryanfox/brainfunction.

The brain

A brainfunction program runs on the brain, which is an extended version of the brainfuck VM. It supports the eight standard brainfuck symbols, instruction pointer, data tape, and data pointer. In addition to this, it has an array of functions. When a brainfunction program is parsed, each line of text in the source file becomes a function. These constitute the function array, starting with the first line of text in position 0. A function is exactly one line of source code. This implies one cannot break up logical units multiple lines as is done in brainfuck.

Functions

Each function has a function pointer, initialized to 0 (the first line in the file). The function pointer is moved by the symbols v and ^, meaning "move the function pointer down" and "move the function pointer up", respectively. Each function also has a local data pointer, data tape, and instruction pointer.

Calling

To call another function, the : symbol is used. When this occurs, the function pointed to by the current function pointer is called. The value in the caller's current data cell is passed to the callee. This argument is placed in cell 0 of the callee's data tape.

Returning

A function returns when the symbol ; is encountered or the function runs out of instructions. When a function returns, the value in the current data cell is returned. The return value is placed in the caller's current data cell.

Evaluation

The brain begins execution in the zeroth function in the function array, and continues until an error is encountered or the function runs out of symbols. All symbols except the 12 specified are ignored. When a function calls another, execution blocks in the caller until the callee returns.

The reference implementation is an interpreter written in python, which can be found here.

Examples

Example hello world:

v:vv:^^:vv:^:v:^:v:^:vv:
>++++++++++[>++++++++++<-]>++++.---.+++++++..+++.                    print hello
>++++++++++++[>++++++++++<-]>-.--------.+++.------.--------.         print world
>++++[>++++++++<-]>.                                                 print space character
>++++[>++++++++<-]>+.                                                print !

When executed, this will print

hello hello world world world!

Example factorial calculator:

+v[,------------------------------------------------:v:v:^^]
[[>+>+<<-]>>-v:<[>[>+>+<<-]>[<+>-]<<-]>>>;]+
[>>+>+<<<-]>>>[<<<+>>>-]<<+>[<->[>++++++++++<[->-[>+>>]>[+[-<+>]>+>>]<<<<<]>[-]++++++++[<++++++>-]>[<<+>>-]>[<<+>>-]<<]>]<[->>++++++++[<++++++>-]]<[.[-]<]<
>++++++++++.<

When executed, this will prompt the user for input (limited to a single character), calculate the factorial of that number, and print the output:

bf > 5
120

Roadmap

Future developments may include real error messages, and perhaps a debugger, to make it easy to inspect the state of the brain. Maybe even zanier symbols!

That's it! Look on my works, ye mighty, and despair!

Needs more lens flare

2015-12-02T12:00:00-06:00

Lens flare is an effect created by light reflecting internally in the lens of a camera. Whether it's good or bad is up to how it's used. Some filmmakers are notorious for this (looking at you, J.J. Abrams).

Inspired by needsmorejpeg, I created an app to take user-uploaded photos and add lens flare to them. Wish your photo had more lens flare? No worries, get it automatically added for you. Just upload your photo and we'll take care of the rest. The site is needsmorelensflare.com. Happy flaring!

Get Play-By-Play NBA SportVU Data

2015-10-30T12:00:00-05:00

Update 06/14/16: It appears stats.nba.com has taken down the SportVU logs. However, neilmj has posted logs from the 2015-16 season on github, for games through January 23.

SportVU is a video system made by STATS Inc, used by teams to track movement of players and the ball on the court. I looked, but couldn't find anywhere describing how to get play-by-play data, so did some investgating myself. The great stats.nba.com has every game here. Clicking on any of the linked values in the table brings up a popup with an option 'Movement':

Inspecting the network request that goes out when clicking on Movement shows us that fires off an AJAX request to http://stats.nba.com/stats/locations_getmoments/?eventid=2&gameid=0021500003 We can infer that eventid=2 refers to field goal attempts, and gameid=0021500003 likely refers to the third game in 2015. Indeed, changing the gameid to 0021400003 returns data for a game dated October 28, 2014, between the Rockets and Lakers.

Once you get the eventid of the type you want, you could theoretically enumerate all the games in a season this way, or across multiple seasons. The response comes in JSON format, and Savvas Tjortjoglou nicely detailed that in this post. Happy number crunching!

Unshorten URLs with unshorten.py

2015-09-11T12:00:00-05:00

Obfuscated urls stink. Being able to hover over a link and see where it goes is a tremendous feature of any modern browser. Use unshorten on your command line to see where they go. The project is at github. Put unshorten.py somewhere on your path and make it executable. You don't even have to leave the .py on the end!

$ git clone git@github.com:ryanfox/unshorten.py.git
$ chmod u+x unshorten.py/unshorten.py
$ mv unshorten.py/unshorten.py /usr/local/bin/unshorten
$ unshorten https://goo.gl/hnpGjq
https://github.com/ryanfox/unshorten.py

Hopefully it's useful.

Multi-line statements with PLY

2015-09-04T12:00:00-05:00

I've been experimenting with PLY and was encountering an obscure error that was surprisingly Google-resistant. I've been building up a language and a REPL to interpret commands. When trying to parse a statement spanning multiple lines, I would get an error {TypeError}Can't convert 'type' object to str implicitly. Debugging revealed the parser was running into an EOF. The usual sources online didn't appear to have any information about it, either.

It turns out the problem wasn't PLY, but Python's input(). It only reads one line at a time. So even if you paste in

def foo(bar):
    print('stuff')
    return bar + 10

the method will only pass along the string 'def foo(bar):. As a workaround, try something like this:

def evaluate(string):
    # your eval function

parser = yacc.yacc()

while True:
    string = ''
    line = input('reap> ')
    while line != '':
        string += line
        line = input()

    print(evaluate(parser.parse(string)))

There's your read-eval-print loop, spanning multiple lines of user input. After your block is finished, hit enter twice to finish the block, and you should be good to go.

US states and words with no letters in common

2015-08-14T12:00:00-05:00

"Ohio is the only state not to share a letter with the word mackerel" my buddy tells me.

Of course, I need to find out what other words don't share any letters with states. So I start hunting for a large English dictionary. It turns out the Brown corpus, included in NLTK has ~1 million words. So I whip up a python script to figure out what state-word combos pass. The code lives here.

Spoiler alert: apparently there's 38969. At least. That's just words from the particular corpus I used. Ohio is the most popular (least popular?) state, clocking in at 1085. Next closest is Mississippi, at 678. Third is Alabama with 599. Ohio doesn't have any A's or E's, so that helps.

NBA winning percentages

2015-04-08T12:00:00-05:00

During a Milwaukee Bucks broadcast on February 11, John McGlocklin mentioned something to the effect of "the first team to get to 100 points usually wins." Intuitively, this makes some sense. If you're the first team to a given score, you have the lead at that point. 100 points is usually a late-game score, and having the lead near the end of the game makes it more likely you are going to win.

I was curious as to whether the data backed this up. I collected play-by-play scoring data for the 2013-2014 season, and ran some analysis on it. First, the number of times the first team to X points won or lost:

That's a little hard to read in the higher-scoring (interesting) portion of the graph. Here's the winning percentages plotted as a function of score:

Some notes:

There were 1319 games played, including playoffs.
Teams win at a .932 clip for scoring 100 first. Not a bad rule of thumb!
On the other hand: at no point having the lead gives you a sub-.500 winning percentage. So you could also say "the first team to 1 point usually wins".
The worst is 1 or 2 points, yielding wins at .547. Even getting to 3 first improves your chances a bit, an extra percent and a half, all the way up to .562.
The highest point total 145. The Rockets beat Lakers on the road on April 8. The game didn't even go to overtime!
There is a curious bump in the ratio at 121. This is an artifact of not many teams scoring that many and still losing - only 3 to 5 losses in that range. Makes for a noisy signal.
No team scoring 128 or more lost.

The Bucks were first to 100 that night, and beat the Kings 111-103 :)

Thanks to basketball-reference.com for the play-by-play data.

Installing OpenCV in a virtualenv on Ubuntu and OSX

2014-10-14T12:00:00-05:00

$ pip install opencv never seems to work quite right. If you get the message

Could not find any downloads that satisfy the requirement opencv
Cleaning up...
No distributions at all found for opencv

There are a handful of answers strewn across the internet about installing OpenCV's Python bindings, but none of them seem to apply to installing them in a virtualenv in Linux. In the interest of collecting all that information in one place, here's what I did to get it running. I'm using:

Ubuntu 14.04 64 bit
Python 2.7.6
OpenCV 2.4.9.0

Update: You may find luck using opencv-python, an unofficial package of the python bindings for OpenCV. If you only need to use the Python interface, you can just pip install opencv-python and should be good to go.

This will install OpenCV, the Python bindings, and Numpy system-wide, but afterward you will be able to use them inside a virtualenv. This assumes you have pip, virtualenv, and virtualenvwrapper installed and properly configured. If you aren't familiar with these, Googling yields many resources, for example this tutorial.

First, install OpenCV's dependencies, per the installation instructions. (Some of these were pre-installed on my system)

$ sudo apt-get install build-essential cmake libgtk2.0-dev pkg-config \
python-dev libavcodec-dev libavformat-dev libswscale-dev{% endhighlight %}

The next part had me tripped up for a little bit. OpenCV doesn't play particularly well with virtualenvs, so numpy needs to be installed on the system Python:

$ sudo pip install numpy

After that, continue to build OpenCV per the instructions. Download the source (I'm using version 2.4.9.0, from here and unzip it in the directory of your choice.

$ unzip opencv-2.4.9.zip
$ cd opencv-2.4.9
$ mkdir build
$ cd build

Configure the make files using cmake. There is a flag required for the Python bindings that I couldn't find in the official documentation, only in StackOverflow questions: BUILD_NEW_PYTHON_SUPPORT. Also note the two trailing periods.

$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local \
-D BUILD_NEW_PYTHON_SUPPORT=ON ..

This will output a lot of text, but if you scroll up you should find a section referring to Python. It will refer to the system Python binary. This is fine. We will set up our virtualenv later.

After the build is configured, time to make the project. This will take a few minutes to run.

$ make
$ sudo make install

After the build completes, you need to set up your virutalenv if you haven't already. Numpy also needs to be installed in the virtualenv.

$ mkvirutalenv opencv
[...]
(opencv) $ pip install numpy

Now that our virtualenv is ready to go, we just need to copy the OpenCV binary into the virtualenv's site-packages directory. It should be somewhere under the location you installed it, mine was in /usr/local/lib/python2.7/dist-packages/cv2.so

(opencv) $ cd lib
(opencv) $ cp /usr/local/lib/python2.7/dist-packages/cv2.so ~/.virtualenvs/opencv/lib/python2.7/site-packages

You can now delete the opencv-2.4.9 directory if you want. The reason we don't want the cv2.so that got built there is the libraries it's linked to. If you run $ ldd cv2.so you'll get something like

libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007fe78eca2000)
libopencv_core.so.2.4 (0x00007fe78e7f9000)
libopencv_flann.so.2.4 (0x00007fe78e585000)
libopencv_imgproc.so.2.4 (0x00007fe78e096000)
(...)

See that first one with an absolute path (/usr/lib/...)? The other ones are linked to copies in the build directory so if you delete them, cv2.so won't have its full feature set, even though it may import correctly.

$ ldd /usr/local/lib/python2.7/dist-packages/cv2.so
libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f9f27e63000)
libopencv_core.so.2.4 => /usr/local/lib/libopencv_core.so.2.4 (0x00007f9f279bb000)
libopencv_flann.so.2.4 => /usr/local/lib/libopencv_flann.so.2.4 (0x00007f9f27746000)
libopencv_imgproc.so.2.4 => /usr/local/lib/libopencv_imgproc.so.2.4 (0x00007f9f27257000)
libopencv_highgui.so.2.4 => /usr/local/lib/libopencv_highgui.so.2.4 (0x00007f9f26e1e000)
(...)

Ahh, much better.

Everything should be good to go:

(opencv) $ python
>>> import cv2
>>>{% endhighlight %}

If you get ImportError: no module named cv2 double check you had the BUILD_NEW_PYTHON_SUPPORT flag set, and that numpy is installed for the system Python. If you get the error message ImportError: numpy.core.multiarray failed to import you need to install numpy in your virtualenv. From inside the virtualenv, (opencv) $ pip install numpy should fix that. I believe that should do the trick.

Update June 14, 2017:

OSX troubleshooting

These tips from a reader may help you troubleshoot errors on OSX Sierra.

If you get Error compiling OpenCV, fatal error: stdlib.h, try adding the cmake flag -DENABLE_PRECOMPILED_HEADERS=OFF

cc1plus: some warnings being treated as errors -Wno-error

error: "if (&annotate_img!=NULL)

This issue has been fixed as of OpenCV version 2.4.13, try upgrading to the latest version of OpenCV 2 or 3.

Good luck with your computer vision!

New host

2014-07-07T12:00:00-05:00

Fox row was down for a while due to some issues with the old web host. It's on a new and improved hosting service now, so there shouldn't be any more problems.

Sweeping the 1-seeds

2014-03-18T12:00:00-05:00

The Bracket is out, and once Virginia was announced as the final 1-seed, I realized Wisconsin may be in a unique position. The Badgers beat both Florida and Virginia during the regular season, and have the possibility to beat both Arizona and Wichita State in the tournament. If they do so, they will have beaten all four 1-seeds this season. I had to find out if anyone ever has. There have been 116 1-seeds since the field expanded to 64 in 1985. Naturally, all the 1-seeds are excluded by virtue of being unable to lose to themselves.

It turns out there have only been 2 teams that have 4 wins against 1-seeds in the same year, and they're both Arizona squads. In 1997, the Wildcats beat North Carolina twice, Kansas, and Kentucky en route to the national championship. They never played Minnesota, the other 1-seed. In an even crazier 2001 season, they came within a game of doing it: they split 2 regular-season games against Illinois before beating them in the tournament, split regular season-games against Stanford, and beat Michigan State in the Final Four. The last 1-seed was Duke, who they lost to in the National Championship. They played 7 games against the 1-seeds that year!

There have been 7 teams with 3 wins against 1-seeds (what was it with the Wildcats those 4 years?): 1985 Illinois 1985 Georgetown 1986 Duke 1991 Duke 1992 Southern California 1992 Indiana 2000 Arizona

The distribution falls off rapidly after that: there have been 64 teams with 2 wins against 1-seeds and 352 with 1 win.

The all-time leaders contain no surprises. If we assume that most of the 1-seeds come from historically "power" conferences, being in the same conference provides more opportunity over the course of a season to play against them. Going deep in the tournament doesn't hurt either.

Duke	26
Arizona	18
North Carolina	17
Indiana	16
Kansas	16
Maryland	16

On the flip side, Wake Forest has the dubious honor of most losses to 1-seeds in a year. In 2002 they also played 7 games against the eventual 1-seeds. They went 0-7, losing to Cincinnati and Kansas once each, Maryland twice, and Duke 3 times. There have been 16 5-loss teams and 73 4-loss teams. In terms of all-time futility, NC State takes the cake. Since 1985, they have a paltry .114 winning percentage against 1-seeds:

NC State	62
Virginia	56
Georgia Tech	54
Clemson	53
Michigan	52

So if the Badgers pull it off, they'll be the first in the 64+ team era to do so.

Stats courtesy Sports-Reference.

Pypeline update

2013-12-04T12:00:00-06:00

I've pushed some changes to the pypeline repository, adding basic stacking functionality. There isn't any registration, it only takes the median of each channel (R, G, B) for each pixel. It's currently way slow, but I suspect there is substantial room for improvement there. Wrangling NEF files has proven more difficult than I anticipated, so currently the state of the art in pypeline is JPGs.

My camera is rated down to 32°F, and nightly lows have been around 0, so I'm scared to take it out into the elements. On the plus side, the stacking works with regular images too! Any particular pixel just needs to have the "right" value for at least half the shots.

Inputs:

And the stacked result:

There is a little ghosting, but quite good considering, I'd say. I am not sure how to get rid of that totally. More pictures should quash the error, but at 5 I would have thought it would wipe out any traces of the marker. Also, a better algorithm should be able to push down the > 50% requirement to only a plurality. Maybe with some sort of clustering of values? I'm also taking the mean of each channel independently, maybe a better way would be to use luminance. In any case, baby steps!

Visual cryptography

2013-11-19T12:00:00-06:00

Inspired by this post at DataGenetics,, I implemented a quick-and-dirty script in python to test it out. The first takes an input image and iterates over it pixel by pixel, splitting it into two output images. Ideally, the outputs are randomly assigned, so it is impossible to recover the original without both outputs. The outputs can be combined to recover the original. Here's an example of it in action:

Original image:

Intermediate images (hopefully look like static):

Final output:

Not perfect, but it is definitely recognizable. The idea can apparently be extended to 9x9 (and 16x16, and 25x25... I presume) images, for a wider-shared secret. In any case, this scheme should make it possible for any number of people to share a secret, but none of them individually can recover it. I uploaded the code here on github.

New PC

2013-11-17T12:00:00-06:00

I've been a little busy lately, but should be able to pick things up again with the new PC I built. Made a time-lapse of assembling it too:

SSL on Fox Row

2013-09-25T12:00:00-05:00

I've just enabled SSL on the site. Technically it's TLS, but the name SSL seems to be sticking. Now you can access it at https://www.foxrow.com (note https). The not-encrypted version should still be available at http://www.foxrow.com. I'm not sure with Heroku hosting the apps if I can get it set up for ergs and weather, still looking into that.

Update: due to Heartbleed, I disabled this. Hopefully with let's encrypt I'll have SSL back.

Another update: Thanks to Let's Encrypt, secure connections are back.

Pillow and NEF files

2013-08-29T12:00:00-05:00

Apparently PIL, and therefore Pillow, do not support Nikon RAW (.NEF) files. From what I can tell, my camera shoots RAWs in a 14-bit grayscale format. With the help of nefarious, I've found a way to get the image data out of my RAW files. Images coming soon!

Project consolidation

2013-08-22T12:00:00-05:00

As part of my ongoing effort to consolidate my various projects onto foxrow.com, I have moved the rowing weather app to weather.foxrow.com. Eventually I hope to get some interactive form of pypeline online as well. I am not sure yet what form that might take. Because RAWs are so big, uploading enough to get a reasonable outcome may be prohibitively bandwith-intensive. Also processing power on the server side may prove too big a bottleneck, but I won't know until I try it.

Setting up Python 3 and pillow

2013-08-03T12:00:00-05:00

I'm using Linux Mint 15 ("Olivia") for my pypeline development - and ran into a little difficulty right off the bat getting pillow installed. Virtualenv and Virtualenvwrapper are pretty much a must-have if you're developing multiple projects, and pip install pillow was failing with messages like Python.h: No such file or directory despite following the instructions to a T. Turns out since I'm using Python 3, sudo apt-get install python-dev doesn't cut it. What you want is sudo apt-get install python3-dev. Run that once and you should be good to go in your virtualenv.

"pypeline v0.0.1 - a python pipeline"

2013-08-03T11:00:00-05:00

Since I plan on taking quite a few night sky pictures, I'd like to automate the process as much as possible. The project is hosted on github here. Right now it's essentially empty, but eventually features I'd like it to have:

Image calibration - support for flat frames, bias frames, dark frames
De-mosaicing
Image registration
Image stacking
An optional GUI

Why roll your own pipeline, when other programs exist?
I'd like to learn more about the process and get some experience with astrophotography.

Why python?
The majority of stacking/alignment applications I've found are windows-only.

Why open source?
The majority of stacking/alignment applications I've found are closed-source, even if they're free as in beer. I like free as in everything.

What formats/algorithms do you support?
Right now, none. Since my camera is a Nikon D5200, the first thing I'll implement is Nikon RAW (NEF) files. Various stacking/averaging algorithms will have to come later.

What's your release schedule?
When I get around to it. There's no hard timetable on anything, but I'd like to get a good chunk implemented sooner rather than later. Got some pictures to process!

Python 3? Really?
Yes, really. Python 3 is the future of the language, and Django and PIL were the two major things holding me back. Well, now Django is py3k compatible, and now we have pillow, the friendly PIL fork. Lucky!

That's all for now, follow the project on github or fork it and contribute!

"Night 2: the big dipper"

2013-08-02T10:00:00-05:00

With one set of exposures under my belt, I adjusted my camera settings a little and tried my hand again. As luck would have it, my window framed the big dipper quite nicely, so I set up in front of that and let it rip.

51 light frames
f/3.5
10 second exposures
ISO 1000
18mm focal length
31 dark frames
31 bias frames - same everything except exposure time: 1/4000th second

Stacked in deep sky stacker and processed in Picasa. I really need to get some better software. This was closer to the horizon than the previous shot, so there was heavy light pollution near the bottom of the frame. After abusing the stacked result pretty severely, I ended up with a recognizable big dipper and black sky. I ended up cropping a fair amount, so the effective focal length is quite a bit longer than 18mm. I think it turned out fairly good, considering:

First light from the Nikon

2013-07-30T12:00:00-05:00

Well, sort of. This is my first attempt at any sort of serious astrophotography. I wasn't aiming for anything in particular, I just wanted to get a feel for my camera. The conditions were subpar to say the least: it was a mildly cloudy night, and I'm near a metropolitan downtown, so the light pollution was terrible. Collection details:

Nikon D5200 w/18-55mm Nikkor lens - RAW (NEF) format
Focal length: 55mm
Exposure time: 10secs
f/11
ISO 100
31 light frames, 11 dark frames
Stacked with Deep Sky Stacker

Considering all that, my results weren't too terrible:

Looks pretty, but that's about it. Technical note: I was having trouble getting my RAWs to align in DSS. I suspect my ISO was way too low for it to detect stars, but after converting to jpg it seemed to do reasonably well.

Fox Row ergs is moving

2013-07-30T10:00:00-05:00

I'm redesigning the site a little. Since I haven't been able to devote much time to Fox Row Ergs, formerly at foxrow.com, I am moving that to ergs.foxrow.com, and this blog is now at foxrow.com. The app itself isn't changing, just my focus on the site is.