GPX to PostGIS, PostGIS to GPX

With ogr2ogr.

export CONN_STRING="host=localhost dbname=DATABASE user=USERNAME password=PASSWORD port=5432"
# Import
ogr2ogr -append -f PostgreSQL PG:dbname=DATABASE_NAME /path/to/your.gpx
# Export
ogr2ogr -f gpx -nlt MULTILINESTRING /path/to/output/tracks.gpx PG:"$CONN_STRING" "tracks(wkb_geometry)"
ogr2ogr -f gpx -nlt MULTILINESTRING /path/to/output/routes.gpx PG:"$CONN_STRING" "routes(wkb_geometry)"
ogr2ogr -f gpx -nlt POINT /path/to/output/waypoints.gpx PG:"$CONN_STRING" "waypoints(wkb_geometry)"

The wkb_geometry references can be replaced with full SQL statements as required.

Short Variable Names in Go

In a recent code review my colleague took issue with the following code.

func Enqueue(properties Properties) (err error) {
	logger := logging.GetLogger(ctx)
	bs, err := json.Marshal(properties)
	if err != nil {
		logger = logger.With().Err(err).Logger()
	} else {
		logger = logger.With().RawJSON("properties", bs).Logger()
	}
        … go on to log some stuff and enqueue the supplied event with the supplied properties …
}

Specifically the question was around whether `bs` was a reasonable name for the variable holding the JSON version of the properties. My counterargument was that short names are better than long names when well understood and/or short in scope. And that Go has a C influence and favors short variable names which you can see in both the standard library and its examples. The Go encoding/json library calls []byte variously src and data (code), b, j and text (examples) – https://golang.org/pkg/encoding/json/

My colleague said it took them longer 0s to understand the var so they called it out as a nit (not a blocker) and that they care more about knowing what is contained within than if it is `[]byte` or not.

I ended up renaming it `propertiesJSON`. It did start a discussion about short variable names in general and in Go in particular. I’m still not sure how I feel about it. I did find some reading that seemed relevant though.

Notes on Programming in C (Variable names)

Variable names in Go should be short rather than long. This is especially true for local variables with limited scope. Prefer c to lineCount. Prefer i to sliceIndex.

Go Code Review Comments (Variable Names)

Local variables. Keep them short; long names obscure what the code does … Prefer b to buffer.

What’s In a Name? (Local Variables)

Share Clipboard Between Terminal (zsh) and OSX

### System-wide Clipboard mostly from https://gist.github.com/welldan97/5127861

pb-kill-line () {
  zle kill-line
  echo -n $CUTBUFFER | pbcopy
}

pb-backward-kill-line () {
  zle backward-kill-line
  echo -n $CUTBUFFER | pbcopy
}

pb-kill-whole-line () {
  zle kill-whole-line
  echo -n $CUTBUFFER | pbcopy
}

pb-backward-kill-word () {
  zle backward-kill-word
  echo -n $CUTBUFFER | pbcopy
}

pb-kill-word () {
  zle kill-word
  echo -n $CUTBUFFER | pbcopy
}

pb-kill-buffer () {
  zle kill-buffer
  echo -n $CUTBUFFER | pbcopy
}

pb-copy-region-as-kill-deactivate-mark () {
  zle copy-region-as-kill
  zle set-mark-command -n -1
  echo -n $CUTBUFFER | pbcopy
}

pb-yank () {
  CUTBUFFER=$(pbpaste)
  zle yank
}

zle -N pb-kill-line
zle -N pb-backward-kill-line
zle -N pb-kill-whole-line
# This is too extreme - I often want to wrangle a commandline then paste into it.
#zle -N pb-backward-kill-word
#zle -N pb-kill-word
zle -N pb-kill-buffer
zle -N pb-copy-region-as-kill-deactivate-mark
zle -N pb-yank

bindkey '^K'   pb-kill-line
bindkey '^U'   pb-backward-kill-line
#bindkey '\e^?' pb-backward-kill-word
#bindkey '\e^H' pb-backward-kill-word
#bindkey '^W'   pb-backward-kill-word
#bindkey '\ed'  pb-kill-word
#bindkey '\eD'  pb-kill-word
bindkey '^X^K' pb-kill-buffer
bindkey '\ew'  pb-copy-region-as-kill-deactivate-mark
bindkey '\eW'  pb-copy-region-as-kill-deactivate-mark
bindkey '^Y'   pb-yank

Calculating Swiss Record Required to Reach Elimination Rounds

Here’s some Python that calculates how many players will reach each record in a Swiss tournament with a Top 8 or similar cut.

from typing import Sequence

# Math from https://www.mtgsalvation.com/forums/magic-fundamentals/magic-general/325775-making-the-cut-in-swiss-tournaments
def swisscalc(num_players: int, num_rounds: int, num_elimination_rounds: int) -> Sequence[int]:
    num_players_in_elimination_rounds = 2 ** num_elimination_rounds
    base = num_players / (2 ** num_rounds)
    num_players_by_losses = [0] * (num_rounds + 1)
    multiplier = 1.0
    total_so_far = 0
    record_required = None
    for losses in range(0, num_rounds + 1):
        wins = num_rounds - losses
        numerator = wins + 1
        denominator = losses
        if denominator > 0:
            multiplier *= (numerator / denominator)
        num_players_by_losses[losses] = base * multiplier
        if not record_required and num_players_in_elimination_rounds:
            total_so_far += num_players_by_losses[losses]
    return num_players_by_losses

Example usage:

$ python3
>>> rounds = 4
>>> r = swisscalc(24, rounds, 3)
>>> for losses in range(len(r)):
...     print(f'{r[losses]} players at {rounds - losses}–{losses}')
... 
1.5 players at 4–0
6.0 players at 3–1
9.0 players at 2–2
6.0 players at 1–3
1.5 players at 0–4

git popclean

If you git stash when you have a bunch of local files that are ignored git stash pop will refuse to un-stash your saved changes. This command cleans that up.


git stash pop 2>&1 | grep already | cut -d' ' -f1 | xargs rm && git stash pop

Make a PUT request in Go (with JSON body)

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"time"
)

type MyRequest struct {
	Name  string `json:"name"`
	Color string `json:"color"`
	Size  int    `json:"size"`
}

type MyResponse struct {
	Status string `json:"status"`
}

func doRequest(httpMethod string, address string, requestBody MyRequest, responseBody *MyResponse) (err error) {
	j, err := json.Marshal(requestBody)
	if err != nil {
		return
	}
	req, err := http.NewRequest(httpMethod, address, bytes.NewReader(j))
	if err != nil {
		return
	}
	req.Header.Set("Content-type", "application/json")
	client := http.Client{Timeout: time.Second * 10}
	resp, err := client.Do(req)
	if err != nil {
		return
	}
	defer resp.Body.Close()
	if resp.StatusCode >= 400 {
		return fmt.Errorf("Request failed with status %d", resp.StatusCode)
	}
	err = json.NewDecoder(resp.Body).Decode(responseBody)
	if err != nil {
		return
	}
	return
}

func main() {
	responseBody := new(MyResponse)
	err := doRequest("PUT", "https://example.com/endpoint", MyRequest{Name: "bakert", Color: "red", Size: 10}, responseBody)
	if err != nil {
		fmt.Println("Failed", err)
	} else {
		fmt.Println("Status", responseBody.Status)
	}
}

Keeping CDN-based js dependencies up to date

For python dependencies you can use a simple requirements.txt file and something like requires.io to keep all your dependencies at latest stable release automatically.

npm and webpack and friends can do a fine job of keeping your js dependencies up to date if you are prepared to bundle them into your js.

But what if you want to use the publicly-hosted CDN versions? There doesn’t seem to be anything available.

This is what I came up with. I’m not in love with it but it works ok so far!

It reads a file that looks like this:

jquery
jquery.hoverIntent
jquery.tablesorter
jquery.tablesorter:jquery.tablesorter.widgets
moment.js
moment-timezone:moment-timezone-with-data
Chart.js
react
react-dom

and emits a file that looks like this:

<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.hoverintent/1.10.0/jquery.hoverIntent.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.tablesorter/2.31.1/js/jquery.tablesorter.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.tablesorter/2.31.1/js/jquery.tablesorter.widgets.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/moment.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/moment-timezone/0.5.26/moment-timezone-with-data.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/Chart.js/2.8.0/Chart.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/react/16.10.2/umd/react.production.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/react-dom/16.10.2/umd/react-dom.production.min.js"></script>

Here’s the code, which lives at https://github.com/PennyDreadfulMTG/Penny-Dreadful-Tools/blob/master/maintenance/client_dependencies.py:

import re
import subprocess
from typing import List

from shared import fetch_tools
from shared.pd_exception import DoesNotExistException

PATH = 'shared_web/templates/jsdependencies.mustache'

def ad_hoc() -> None:
    tags = [fetch_script_tag(library) + '\n' for library in get_dependencies()]
    output = ''.join(tags)
    write_dependencies(output)
    send_pr_if_updated()

def get_dependencies() -> List[str]:
    f = open('shared_web/jsrequirements.txt', 'r')
    return [line.strip() for line in f.readlines()]

def write_dependencies(s: str) -> None:
    f = open(PATH, 'w')
    f.write(s)

def send_pr_if_updated() -> None:
    subprocess.call(['git', 'add', PATH])
    if subprocess.call(['git', 'commit', '-m', 'Update client dependencies.']) == 0:
        subprocess.call(['git', 'push'])
        subprocess.call(['hub', 'pull-request', '-b', 'master', '-m', 'Update client dependencies.', '-f'])

def fetch_script_tag(entry: str) -> str:
    parts = entry.split(':')
    library = parts[0]
    file = parts[0] if len(parts) == 1 else parts[1]
    info = fetch_tools.fetch_json(f'https://api.cdnjs.com/libraries/{library}')
    version = info.get('version')
    if not version and library.lower() != library:
        library = library.lower()
        info = fetch_tools.fetch_json(f'https://api.cdnjs.com/libraries/{library}')
        version = info.get('version')
    if not version:
        raise DoesNotExistException(f'Could not get version for {library}')
    path = None
    for a in info['assets']:
        if a.get('version') == version:
            for f in a['files']:
                if minified_path(f, file):
                    path = f
                    break
                if unminified_path(f, file):
                    path = f
    if not path:
        raise DoesNotExistException(f'Could not find file for {library}')
    return f'<script defer src="//cdnjs.cloudflare.com/ajax/libs/{library}/{version}/{path}"></script>'

def minified_path(path: str, library: str) -> bool:
    return test_path(path, library, '.min')

def unminified_path(path: str, library: str) -> bool:
    return test_path(path, library)

def test_path(path: str, library: str, required: str = '') -> bool:
    # CommonJS libs get us the error 'require is not defined' in the browser. See #6731.
    if 'cjs/' in path:
        return False
    name_without_js = library.replace('.js', '')
    regex = fr'{name_without_js}(.js)?(.production)?{required}.js

    return bool(re.search(regex, path, re.IGNORECASE))

Set Up a New Mac

Updated from a few years ago: http://bluebones.net/2015/12/things-i-absolutely-must-do-on-a-new-mac/

  • Upgrade to latest OSX.
  • Restore home directory from Backblaze. (Or at least dotfiles, .ssh dir, .vim dir, bin directory, ~/Application Support/Alfred 4/*, pwd.dsv.gpg, business, travel)
  • Dock: remove everything, hiding on, magnification on.
  • Sublime Text 3 (and add license key, set “trim_trailing_white_space_on_save”: true in sublime prefs).
  • Solarized theme for Sublime Text 3 (after installing Package Control).
  • subl commandline sublime.$ ln -s “/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl” ~/bin/subl
  • Sublime git ignorer (to have ST3 respect .gitignore in searches, etc.) via Package Control.
  • Hide Desktop icons.
    $ defaults write com.apple.finder CreateDesktop false
    $ killall Finder
  • Alfred (and give it Accessibility access, add license key).
  • Solarized theme for Terminal.
  • “Use Option as Meta Key” in Terminal to make left Alt work as Esc.
  • Use Ctrl-f7 or the Keyboard, Shortcuts in System Preferences to give Full Keyboard Access so that tab takes you to every button in a dialog, etc.
  • Discord.
  • Homebrew. (Installs OSX commandline tools.)
  • Change all .txt to open with Sublime Text.
  • Show hidden files in Finder. Cmd-Shift-.
  • WhatsApp.
  • 1Password.
  • Wine + MTGO: https://github.com/pauleve/docker-mtgo/wiki/macOS:-installing-MTGO-using-Wine
  • Chrome. (Log in to Chrome to get uBlock Origin and other extensions.)
  • Skitch.
  • mysql/mariadb (via homebrew).
  • rtm-cli (via homebrew).
  • Backblaze.