bluebones.net – Page 5 – Adventures in Computer Programming

May 1, 2020

Make a PUT request in Go (with JSON body)

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"time"
)

type MyRequest struct {
	Name  string `json:"name"`
	Color string `json:"color"`
	Size  int    `json:"size"`
}

type MyResponse struct {
	Status string `json:"status"`
}

func doRequest(httpMethod string, address string, requestBody MyRequest, responseBody *MyResponse) (err error) {
	j, err := json.Marshal(requestBody)
	if err != nil {
		return
	}
	req, err := http.NewRequest(httpMethod, address, bytes.NewReader(j))
	if err != nil {
		return
	}
	req.Header.Set("Content-type", "application/json")
	client := http.Client{Timeout: time.Second * 10}
	resp, err := client.Do(req)
	if err != nil {
		return
	}
	defer resp.Body.Close()
	if resp.StatusCode >= 400 {
		return fmt.Errorf("Request failed with status %d", resp.StatusCode)
	}
	err = json.NewDecoder(resp.Body).Decode(responseBody)
	if err != nil {
		return
	}
	return
}

func main() {
	responseBody := new(MyResponse)
	err := doRequest("PUT", "https://example.com/endpoint", MyRequest{Name: "bakert", Color: "red", Size: 10}, responseBody)
	if err != nil {
		fmt.Println("Failed", err)
	} else {
		fmt.Println("Status", responseBody.Status)
	}
}

November 28, 2019

Keeping CDN-based js dependencies up to date

For python dependencies you can use a simple requirements.txt file and something like requires.io to keep all your dependencies at latest stable release automatically.

npm and webpack and friends can do a fine job of keeping your js dependencies up to date if you are prepared to bundle them into your js.

But what if you want to use the publicly-hosted CDN versions? There doesn’t seem to be anything available.

This is what I came up with. I’m not in love with it but it works ok so far!

It reads a file that looks like this:

jquery
jquery.hoverIntent
jquery.tablesorter
jquery.tablesorter:jquery.tablesorter.widgets
moment.js
moment-timezone:moment-timezone-with-data
Chart.js
react
react-dom

and emits a file that looks like this:

<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.hoverintent/1.10.0/jquery.hoverIntent.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.tablesorter/2.31.1/js/jquery.tablesorter.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/jquery.tablesorter/2.31.1/js/jquery.tablesorter.widgets.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/moment.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/moment-timezone/0.5.26/moment-timezone-with-data.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/Chart.js/2.8.0/Chart.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/react/16.10.2/umd/react.production.min.js"></script>
<script defer src="//cdnjs.cloudflare.com/ajax/libs/react-dom/16.10.2/umd/react-dom.production.min.js"></script>

Here’s the code, which lives at https://github.com/PennyDreadfulMTG/Penny-Dreadful-Tools/blob/master/maintenance/client_dependencies.py:

import re
import subprocess
from typing import List

from shared import fetch_tools
from shared.pd_exception import DoesNotExistException

PATH = 'shared_web/templates/jsdependencies.mustache'

def ad_hoc() -> None:
    tags = [fetch_script_tag(library) + '\n' for library in get_dependencies()]
    output = ''.join(tags)
    write_dependencies(output)
    send_pr_if_updated()

def get_dependencies() -> List[str]:
    f = open('shared_web/jsrequirements.txt', 'r')
    return [line.strip() for line in f.readlines()]

def write_dependencies(s: str) -> None:
    f = open(PATH, 'w')
    f.write(s)

def send_pr_if_updated() -> None:
    subprocess.call(['git', 'add', PATH])
    if subprocess.call(['git', 'commit', '-m', 'Update client dependencies.']) == 0:
        subprocess.call(['git', 'push'])
        subprocess.call(['hub', 'pull-request', '-b', 'master', '-m', 'Update client dependencies.', '-f'])

def fetch_script_tag(entry: str) -> str:
    parts = entry.split(':')
    library = parts[0]
    file = parts[0] if len(parts) == 1 else parts[1]
    info = fetch_tools.fetch_json(f'https://api.cdnjs.com/libraries/{library}')
    version = info.get('version')
    if not version and library.lower() != library:
        library = library.lower()
        info = fetch_tools.fetch_json(f'https://api.cdnjs.com/libraries/{library}')
        version = info.get('version')
    if not version:
        raise DoesNotExistException(f'Could not get version for {library}')
    path = None
    for a in info['assets']:
        if a.get('version') == version:
            for f in a['files']:
                if minified_path(f, file):
                    path = f
                    break
                if unminified_path(f, file):
                    path = f
    if not path:
        raise DoesNotExistException(f'Could not find file for {library}')
    return f'<script defer src="//cdnjs.cloudflare.com/ajax/libs/{library}/{version}/{path}"></script>'

def minified_path(path: str, library: str) -> bool:
    return test_path(path, library, '.min')

def unminified_path(path: str, library: str) -> bool:
    return test_path(path, library)

def test_path(path: str, library: str, required: str = '') -> bool:
    # CommonJS libs get us the error 'require is not defined' in the browser. See #6731.
    if 'cjs/' in path:
        return False
    name_without_js = library.replace('.js', '')
    regex = fr'{name_without_js}(.js)?(.production)?{required}.js

    return bool(re.search(regex, path, re.IGNORECASE))

July 14, 2019March 20, 2022

Set Up a New Mac

Updated from a few years ago: http://bluebones.net/2015/12/things-i-absolutely-must-do-on-a-new-mac/

Upgrade to latest OSX.
Restore home directory from Backblaze. (Or at least dotfiles, .ssh dir, .vim dir, bin directory, ~/Library/Application Support/Alfred 4/*, pwd.dsv.gpg, business, travel)
Dock: remove everything, hiding on, magnification on.
Sublime Text 3 (and add license key, set “trim_trailing_white_space_on_save”: true in sublime prefs).
Solarized theme for Sublime Text 3 (after installing Package Control).
subl commandline sublime.$ ln -s “/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl” ~/bin/subl
Sublime git ignorer (to have ST3 respect .gitignore in searches, etc.) via Package Control.
Hide Desktop icons.
$ defaults write com.apple.finder CreateDesktop false
$ killall Finder
Alfred (and give it Accessibility access, add license key).
Solarized theme for Terminal.
“Use Option as Meta Key” in Terminal to make left Alt work as Esc.
Use Ctrl-f7 or the Keyboard, Shortcuts in System Preferences to give Full Keyboard Access so that tab takes you to every button in a dialog, etc.
Discord.
Homebrew. (Installs OSX commandline tools.)
Change all .txt to open with Sublime Text.
Show hidden files in Finder. Cmd-Shift-.
WhatsApp.
1Password.
Wine + MTGO: https://github.com/pauleve/docker-mtgo/wiki/macOS:-installing-MTGO-using-Wine
Chrome. (Log in to Chrome to get uBlock Origin and other extensions.)
Skitch.
mysql/mariadb (via homebrew).
rtm-cli (via homebrew).
Backblaze.

February 22, 2019

Outlived

outlived.bluebones.net – The site that tells you whom you have outlived.

January 6, 2019

Read Properties File in Spring with Kotlin

import java.io.FileInputStream
import java.util.Properties
import org.springframework.util.ResourceUtils

object Configuration {
    private val properties: Properties = Properties()

    val databaseHost: String?
    val databaseName: String?
    val databaseUser: String?
    val databasePassword: String?
    val securitySecret: String

    init {
        val file = ResourceUtils.getFile("classpath:app.properties")
        val stream = FileInputStream(file)
        properties.load(stream)
        databaseHost = properties.getProperty("database.host")
        databaseName = properties.getProperty("database.name")
        databaseUser = properties.getProperty("database.user")
        databasePassword = properties.getProperty("database.password")
        securitySecret = properties.getProperty("security.secret")!!
    }
}

December 22, 2018

Education

“Schools are places where you send your children to be brought up by other children.”

– Alan Watts

December 21, 2018December 21, 2018

Random rows in MySQL with Reasonable Performance

Almost every answer on Stack Overflow for this is terrible on moderately complex or large real data – https://stackoverflow.com/questions/4329396/mysql-select-10-random-rows-from-600k-rows-fast.

Bad: ORDER BY RAND() LIMIT N

This means executing your query for the entire resultset and then ordering it and then chopping off the number you need. This can have truly dire performance.

Bad: SELECT MAX(id) FROM table
Don’t pick random numbers from 1 to MAX(id) – deleted rows will be null or result in you getting less rows than you want. Who says your id is even sequential/numeric?

OK: SELECT id FROM table

Then pick N at random (removing those already chosen if you don’t want duplicates) from the resultset in your chosen programming language.

If you just want one row from the db you can do this in SQL only as shown here: https://stackoverflow.com/a/31066058/375262. Doing N unique rows this way is left as an exercise for the reader.

If you have a gargantuan resultset even SELECT COUNT(*) might be slow. What would you do then?

December 5, 2018December 5, 2018

How to call a method on an object originating in Java called “in”, which is a kotlin reserved word

Backticks around the reserved word.

I found this working with JOOQ’s implementation of the SQL IN predicate. You can’t write myJavaLibObject.in(…) in Kotlin. You’ll get “Element expected” because in is a reserved word in Kotlin. Instead you need backticks: myJavaLibObject.`in`(…) will do the trick.

December 3, 2018

Convert java.sql.Timestamp to java.time.OffsetDateTime

Hard to believe but I think this is the simplest you can manage:

long millisSinceEpoch = timestamp.getTime();
Instant instant = Instant.ofEpochMilli(millisSinceEpoch)
OffsetDateTime dt = OffsetDateTime.ofInstant(instant, ZoneId.of("UTC"))

Yikes.

December 1, 2018December 1, 2018

Top 50 Rising Programming Technologies

(Based on Stack Overflow tag count and upward trend as proportion of Stack Overflow questions. So it might just be the 50 most difficult-to-learn rising programming technologies!)

The List

Python
Interpreted high-level programming language for general-purpose programming
React
JavaScript library for building user interfaces
Laravel
PHP web framework (“for web artisans”)
Pandas
Data structures and data analysis tools for Python
TypeScript
Superset of JavaScript that adds static typing (“JavaScript that scales”)
Amazon Web Services
On-demand cloud computing platform
API
The interface between two programs.
Azure
Cloud computing service
Powershell
Commandline shell and associated scripting language
Firebase
Mobile and web application development platform
Selenium
Spring Boot
Docker
React Native
DataFrame
Unity 3D
Elasticsearch
Matplotlib
Go
Jenkins

Selenium Web Driver
Gradle
Machine Learning
Amazon S3
vue.js
ggplot2
Flask
ASP.NET Core
npm
Webpack

Mongoose
tkinter
Google Apps Script
Web Scraping
Spring Security
filter
https
Woo Commerce
Xamarin Forms
Web Socket

Android Recycler View
Kotlin
Redux
Google Sheets
Excel Formula
SASS
Hive
Java 8
Redis
CMake

The Top Ten

Chart of the top 10:

The Top Ten Without Python

Python dwarfs everything else so here’s a look without Python:

The Nearly Men

These tags were eliminated from the list solely on the basis of a 2018-only downward trend: R (would have been 2nd), Node.js (2nd), PostgreSQL (4th), numpy (12th), Express (14th), Apache Spark (14th), Tensorflow (18th), nginx (20th), Github (21st), Amazon EC2 (31st), ECMAScript 6 (39th), ffmpeg (46th)

Programming Languages That Make the List

Python
TypeScript
Go
Kotlin

A Rising Python Lifts All Python Libraries

python-3.x actually makes second place on the list but I rolled it into Python rather than make a redundant entry.

Django and Django Models were eliminated from the list despite being on an upward trend because they have not yet exceeded their previous peak in 2010. This resurgence, Pandas in fourth place and the presence of tkinter on the list speaks to the general rising of Python.

Methodology

Load all tags on StackOverflow by count descending.

Put each of them into StackOverflow Trends and judge by eye if they are currently trending up.

Where Do These Technologies Sit in the Overall List?

In the whole list of tags by count Python is sixth overall, Firebase (10th place in this list) is 90th, and CMake (50th) is 442nd.