cesarkina.com

cktop - from /proc to gopsutil and back

cesar@cesarkina.com (Cesar Kina) — Tue, 19 May 2026 00:00:00 +0000

Notes on the rationale behind a recent decision I made for cktop [1] (my cli tool to monitor machines CPU, memory and processes) to start reading stats from /proc instead of using gopsutil. The story is a full circle: the original version of the tool was already reading /proc [2], then I switched to gopsutil, and then I went back to /proc. Here is why.

Context

I started to build cktop because I wanted to:

know what my machine (a PC running Debian) is running and stats about resources
build a tool I can use on a daily basis, produce natural/organic feedback, friction to build cognitive muscles outside work
be exposed to topics I am interested in: observability, OS, golang

Why gopsutil

gopsutil was my go-to library to get what I needed: stats CPU, memory and processes, and was great because it also supports other OS like Windows, so I could even test my binaries in my wife’s laptop, cool.

The issue

Nothing wrong with gopsutil, it does what it’s supposed to do. The problem was that I was not getting the feedback and learning I wanted, because gopsutil abstracts all that knowledge for me: how /proc is structured, what fields to read, how to parse them, how to interpret them. I destroyed my learning experience unintentionally. That is the main reason I decided to go back to read directly /proc.

The move

Shortly after I realized I was not getting the knowledge I originally intended to, I decided to go back to read /proc directly again [3]. A few things came up:

Clean Architecture and FCIS (Functional Core, Imperative Shell) [4] paid off here. The domain layer defines three interfaces, MemoryReader, ProcessReader, and CPUReader, and they know nothing about where the data actually comes from. Both the gopsutil adapters and the new /proc adapters implement those same interfaces. The only place in the codebase that knows which one to use is main.go. So the full migration ended up being 3 lines changed there, just swapping the adapter names. The UI layer, the domain logic, the collector, none of that had to change. I could also leave the gopsutil adapter in place for a while if I needed it. I decided to remove the dead code at the end.
I dropped multi-OS support, so no more binaries on my wife’s laptop or my neighbor’s MacBook. But cktop is a learning tool for me, and my interests are Linux, Go, and systems observability, so Linux-only aligns better with what I actually wanted to build.
I just learned that /proc is actually NOT a directory, it is defined as a “pseudo file-system” where we can find statistic data written by the Linux kernel: CPU, memory, processes [5]. Being a “pseudo file-system”, we can read files there as if they were real directories and files, that’s why cktop can use plain os.Open [6].
I found things I didn’t know, things gopsutil was handling silently. For example, I had no idea that CPU usage is not a single number you just read, it requires two consecutive reads and computing the difference between them. Or that memory “available” and memory “free” are not the same thing. Reading /proc directly made all of that visible, and that’s exactly what I was looking for from the beginning.

Conclusions

Using gopsutil saved me time getting features done. But it also blocked the learning I originally wanted, and I didn’t realize it until I was already deep in it. This time, learning > features.

Also, because the data source was behind an interface, I could swap the whole implementation and the rest of the app didn’t even notice. That’s Clean Architecture and FCIS doing their job, not as a concept but as something practical that actually saved me time when I needed it.

cktop still is my personal side project I use to learn concepts about what’s interesting to me these days: Go + Observability + Systems + Linux. I have some more ideas to introduce to this tool soon, they relate to metrics and monitoring and how to persist stats, which is also huge for me.

Finding the right architecture for my CLI tool - Clean Architecture and FCIS

cesar@cesarkina.com (Cesar Kina) — Thu, 14 May 2026 00:00:00 +0000

While working on cktop (my own htop tool to monitor my machines) [1], I noticed some architectural friction while implementing features. This tool is just a CLI that shows process data (CPU, memory usage) that’s running on a machine. So it’s not an enterprise application that needs an ultra clever abstraction.

However, very soon I ended up myself mixing “core” logic with UI logic, for example, logic to sort processes by PID, and how I present the sorted slice in terminal. That smelled really bad. So I decided to apply some form of Clean Architecture, where the domain code doesn’t know anything about concrete external dependencies. That alleviated some of the friction, boundaries started to be clearer. Yet, UI started to grow in lines of code, and boundaries started to be less clear again.

Clean Architecture, in my opinion, for a project like this can feel mentally exhausting, like before adding a new logic or feature, I have to think about how it will fit best across multiple layers, dependency inversion, and all the nice words about it, while I like the concepts, it was just slowing me down, again, mentally.

So I found another concept, which apparently isn’t new, it’s been there for more than a decade, but it is certainly new to-me. “Functional Core, Imperative Shell” [2]. Citing:

A functional core should contain pure, testable business logic, which is free of side effects (such as I/O or external state mutation). It operates only on the data it is given.

An imperative shell is responsible for side effects, like database calls and sending emails. It uses the functions in your functional core to perform the business logic.

What I found useful to understand is that FCIS and Clean Architecture aren’t really alternatives, they operate at different levels. CA is about how you organize layers and manage dependency direction. FCIS is about whether functions inside those layers are pure or impure. You can use FCIS within a Clean Architecture.

So that’s basically what I tried to follow within cktop, extracted some “domain/core” logic from the UI to a pure function that can be easily tested and invoked from anywhere. It was easy to determine which part of the logic can be extracted and be “pure”, so the mental-heavy-friction I usually have with Clean Architecture wasn’t there. I am not saying FCIS is going to be always my go-to architectural decision over Clean Architecture, in fact, from what I liked, is that they can co-exist, but definitely a refreshing mental model to not overcomplicate things when shouldn’t. FCIS gave me a clear, compiler-enforceable boundary with no ceremony. For a project this size, that’s enough.

Below is the thought process that I documented while I was trying to ease the friction in cktop [3]:

And this is one of the “pure functions” I extracted from the UI code to the domain layer. No receivers, just domain inputs and outputs being processed.

 // FilterProcesses returns the subset of procs whose fields contain query (case-insensitive).
// Returns procs unchanged if query is empty.
func FilterProcesses(procs []Process, query string) []Process {
if query == "" {
return procs
}
q := strings.ToLower(query)
var out []Process
for _, p := range procs {
if strings.Contains(strings.ToLower(fmt.Sprintf("%d %d %s %s %d", p.Pid, p.Ppid, p.Username, p.Cmdline, p.Rss)), q) {
out = append(out, p)
}
}
return out
}

Yes, this is just a strings.Contains call. The point isn’t the complexity of the extracted function. It’s learning to recognize which code has no business knowing about the terminal, the bubbletea framework, or any side effect at all. That recognition is what FCIS gives you.

How to show a clock in terminal prompt

cesar@cesarkina.com (Cesar Kina) — Mon, 24 Jan 2022 00:00:00 +0000

How to show a clock in your terminal prompt.

Goal

In terminal, some times I want (and need) to see the exact time when I am running a command. So, two things to accomplish:

Show the current time in terminal, this should work as an actual clock moving. E.g. 13:32:54, then 13:32:55, and so on.
Take a snapshot of the exact time when a command was run. E.g. I ran this command at 13:32:55 exactly, so that I can revisit my terminal and see when that happened.

Approach

Avoid bloated plugins / solutions, since all I want to have is literally a clock in my terminal prompt, the simpler the better, considering I run shell with zsh on Ubuntu 20.04.3 LTS.

After some research, found this in askubuntu.com: https://askubuntu.com/a/360172 . This answer is well explained, I just adjusted some of the styles that worked for me.

The script

This is the simplified version of the script in my .zshrc file that should hopefully help my future me (and probably others) to see what’s needed:

# "If the PROMPT_SUBST option is set, the prompt string is first subjected to parameter expansion, command substitution and arithmetic expansion."
# See: https://zsh.sourceforge.io/Doc/Release/Prompt-Expansion.html
setopt PROMPT_SUBST
# Clock format
# See: Same link above
PROMPT='%D{%H:%M:%S} $ '
# Set time out of 1 second
# See: https://zsh.sourceforge.io/Doc/Release/Parameters.html
TMOUT=1
# Executed every TMOUT seconds. 1 second in this example
TRAPALRM() {
# Reset the prompt, so that the clock acts like it's "moving"
# See: https://zsh.sourceforge.io/Doc/Release/Zsh-Line-Editor.html#index-reset_002dprompt
zle reset-prompt
}

Results

In terminal, clock should show like so:

13:49:00 $ date
Mon 24 Jan 2022 01:49:00 PM -05
13:49:04 $ date
Mon 24 Jan 2022 01:49:04 PM -05

Reference

Cheat Sheet - Java

cesar@cesarkina.com (Cesar Kina) — Thu, 13 Jan 2022 00:00:00 +0000

My Cheat Sheet for: Java.

Gathering quick references I used to search over and over again. Good to have them handy and supposed to (slowly) grow.

Which Functional Interface fits better for my use case? This answer from stackoverflow gives exactly what my brain can’t retain and forces me to read this more than once: https://stackoverflow.com/a/62434813
How can I prettify this one line of java object data? : https://codebeautify.org/javaviewer

Before prettify

ClassA{id=1, name='My ClassA', classB=[ClassB{id=100, name='My ClassB (1)'}, ClassB{id=101, name='My ClassB (2)'}]}

After prettify

ClassA {
id = 1, name = 'My ClassA', classB = [ClassB {
id = 100, name = 'My ClassB (1)'
}, ClassB {
id = 101, name = 'My ClassB (2)'
}]
}

See other cheatsheets here

Connect to MySQL via terminal with mycli

cesar@cesarkina.com (Cesar Kina) — Fri, 04 Jun 2021 00:00:00 +0000

Quick note on how to connect to MySQL via terminal.

First option, I went to MySQL docs, found mysql command can be installed from here: https://dev.mysql.com/downloads/mysql/. The only “problem” with that is it would also install a MySQL Server in my machine, and I just wanted to have the client.

Found mycli:

A command line client for MySQL that can do auto-completion and syntax highlighting.

Followed the installation instructions, and.. connected!

Big thanks to the people involved in this excellent cli!

Notes

Tested on:

Ubuntu 20.04.2 LTS
Python 3.8.5

How to set timezone in Intellij IDEA

cesar@cesarkina.com (Cesar Kina) — Fri, 28 May 2021 00:00:00 +0000

Quick note on how to set timezone in Intellij IDEA.

Steps

Step 1: Create a simple java project (main() method to just print the current date-time for testing)
Step 2: Set timezone in Intellij (will set timezone via JVM options)

Step 1: Create a simple java project

Create a Java project with only one single java class called App.java.
Create its method main to be executable
Print the current date and time in console

import java.util.Date;
public class App {
public static void main(String[] args) {
System.out.println(new Date());
}
}

Run the app (right click on the class > Run App.main), current date and time reflects my timezone in the console (Peru in my case):

Fri May 28 16:38:08 PET 2021

Step 2: Set timezone in Intellij

Open Run/Debug Configurations and click on Add VM options

Under VM options, type the timezone you want. Example: -Duser.timezone="UTC" if you want UTC.

Run the application again, current date and time should now reflect UTC in the console:

Fri May 28 21:43:39 UTC 2021

Notes

Tested with:

Intellij IDEA v2021.1.1
Java 8

Cheat Sheet - Python

cesar@cesarkina.com (Cesar Kina) — Sat, 22 May 2021 00:00:00 +0000

My Cheat Sheet for: Python.

Gathering quick references I used to search over and over again. Good to have them handy and supposed to (slowly) grow.

Create virtual environment with name venv

python3 -m venv venv

Activate virtual environment venv

source venv/bin/activate

Exit virtual environment

deactivate

Save packages to requirements.txt

pip freeze > requirements.txt

Install packages from requirements.txt

pip install -r requirements.txt

Clean ALL packages. Credits: https://stackoverflow.com/questions/11248073/what-is-the-easiest-way-to-remove-all-packages-installed-by-pip

pip freeze | xargs pip uninstall -y

pipdeptree: Useful python command to see packages in dependency tree format

# Command:
$ pipdeptree
# Output:
commonmark==0.9.1
Jinja2==2.11.3
- MarkupSafe [required: >=0.23, installed: 1.1.1]
pipdeptree==2.0.0
- pip [required: >=6.0.0, installed: 21.0.1]
python-frontmatter==0.5.0
- PyYAML [required: Any, installed: 5.4]
- six [required: Any, installed: 1.15.0]
setuptools==54.2.0

See other cheatsheets here

Tinylog - Simple example with Rolling File Writer

cesar@cesarkina.com (Cesar Kina) — Tue, 26 Jan 2021 00:00:00 +0000

This is a simple example of how to configure a Rolling File Writer with Tinylog.

Steps

Create a java application that simply writes a log using Tinylog (our “Hello World”)
Setup a basic Rolling File Writer
Customize our Rolling File Writer
Test every significant change during the process so that we can understand each piece

Requirements

Java 6 or higher
Maven
IDE of your preference (I’ll use Intellij IDEA)

Create a java application

Create the pom.xml and add the two dependencies required to work with Tinylog.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.ckinan</groupId>
<artifactId>tinylog</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.tinylog</groupId>
<artifactId>tinylog-api</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.tinylog</groupId>
<artifactId>tinylog-impl</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
</project>

Create a java class, I’ll name it App.java with a main method that will just log a Hello World (for now).

package com.ckinan;
import org.tinylog.Logger;
public class App {
public static void main(String[] args) {
Logger.info("Hello world");
}
}

Run the App class, you should see this in your console:

2021-01-26 17:37:28 [main] com.ckinan.App.main()
INFO: Hello world

Ok, so far we just wrote almost the same setup from the Getting Started documentation. Now, let’s configure the Rolling File Writer.

Setup a Rolling File Writer

Create a file tinylog.properties in src/main/resources:

writer = rolling file
writer.file = log/app_{count}.log

Run the App class, you should see:

A new folder created by Tinylog in the root path of your java application ./log
A log file created by Tinylog located in ./log/app_0.log
The content of the log file should be our Hello World:

2021-01-26 17:59:44 [main] com.ckinan.App.main()
INFO: Hello world

If you run the java application a second time, a new log file will be created, this time with name ./log/app_1.log. That’s expected because with our initial configuration in tinylog.properties it’s taking default values about how new log entries will be written in the log files. However, we can customize the way these log files are created, their format, and policies. We will do that in a bit.

Ok, so far we created the Rolling File Writer with just the required properties in it. Now, let’s customize our writer so that we can see a bit more what we can do with it.

Customize our Rolling File Writer

Let’s we have the following needs:

Only consider logging with level INFO
Custom format date like 2021-01-26 20:31:57.729
Class name and line number specifying where exactly the logging came from
The actual message log
The process ID of the application
The thread where the logging came from
Each log file (being rotated by Tinylog) should have maximum 50KB of size
Rotate to maximum 5 log files

All the above information (and more) can be achieved by the logging configuration that Tinylog provides. For more details about this configuration, go to https://tinylog.org/v2/configuration.

In order to meet the said criteria, let’s update our tinylog.properties file:

writer = rolling file
writer.level = info
writer.format = {date: yyyy-MM-dd HH:mm:ss.SSS} {pid}/{thread-id} {class}:{line} {level}: {message}
writer.file = log/app_{count}.log
writer.charset = UTF-8
writer.policies = size: 50KB
writer.backups = 5
writer.append = true

Also, let’s update our java application to print more entries in the log, so that the log files can be rotated:

package com.ckinan;
import org.tinylog.Logger;
public class App {
public static void main(String[] args) {
for(int i=0; i<6000; i++) {
Logger.info(i);
}
}
}

Run the App class, you should see:

Five files with maximum 50KB of size per each. We intentionally forced to create six files so that we can see that app_0.log and app_1.log log files where removed given our configuration to have maximum 5 log files as backup.

$ ls -l ./log
total 512
drwxr-xr-x 7 ckinan staff 224B Jan 26 20:41 .
drwxr-xr-x 9 ckinan staff 288B Jan 26 20:41 ..
-rw-r--r-- 1 ckinan staff 50K Jan 26 20:41 app_2.log
-rw-r--r-- 1 ckinan staff 50K Jan 26 20:41 app_3.log
-rw-r--r-- 1 ckinan staff 50K Jan 26 20:41 app_4.log
-rw-r--r-- 1 ckinan staff 50K Jan 26 20:41 app_5.log
-rw-r--r-- 1 ckinan staff 45K Jan 26 20:41 app_6.log

Log entries should show our custom date format, the process id, thread name, class name and line number. All that according to the format we specified in tinylog.properties under writer.format.

2021-01-26 20:41:45.708 2990/1 com.ckinan.App:9 INFO: 1754

Final thoughts

When it comes to initial setup, I found Tinylog very easy to follow. Their Getting Started guide shows all what I needed (dependencies, requirements and the “Hello World”). Later, when I had to setup the log files rotation, the documentation was also pretty clear on what I needed. I found this logging framework while working on a side project and I feel it was an excellent choice because of its simplicity to cover all my needs regarding what I could do with the Rolling File Writer.

My own static site generator

cesar@cesarkina.com (Cesar Kina) — Mon, 11 Jan 2021 00:00:00 +0000

I am moving from using Gatsby to start generating this site with my own script generator.

The problem

I’d say, there is not an actual problem with Gatsby. In fact, everything I’ve done so far with Gatsby went pretty well. I just want to learn what it takes to do it from scratch, being my site a very simple blog without any complex data states or plugins everywhere.

Scope

This new generator should:

Create pages based on templates and layouts
Support markdown, because I want to keep the current format of my articles
Support frontmatter, because again, all the metadata of my articles should not change at all

Out of the scope

I don’t really want to complicate myself with javascript. So, all my pages will be generated in plain HTML files.

The tool

I just wrote a very simple python script that uses:

jinja2: To generate my pages with templates
~~commonmark~~ mistune: To support markdown for my pre-existing (and new) articles
frontmatter: To support frontmatter and read the metadata of my articles

The logic is actually simple, it just reads a directory (./blog) which contains all the markdown files. Then iterates them to render the list of posts in the main page. After that generates the About page and all the actual articles. All that is done by using the above libraries. All templates are read from a second folder ./resources. Finally, the static files are created in the ./public, which is overwritten everytime the python script runs.

Local testing

For local environment, and since it is all static files, I’m using a Chrome extension called Web Server for Chrome which is simple to use and accomplish my use case. I just need to locate the ./public folder and open http://127.0.0.1:8887 in my Browser. It also has the ability to open the site in your local network, so that I can test it in my mobile device.

Deploy in “production”

EDIT: On June 2021, I moved this site from Netlify to Firebase.

~~Since I’m using Netlify to host my site, there are few things I had to do to have Netlify using my brand new static site generator:~~

~~1) runtime.txt: Needed to specify the python version that should be used to run my script. In my case I set it 3.7.~~ ~~2) netlify.toml: Included a rule to redirect all “not found” resources to a custom 404.html page.~~ ~~3) Changed the “Build command” to have python generate.py, so that Netlify will ran that command to actually build the site per my requirement.~~ ~~4) Changed the “Publish directory” to be ./public, since I am giving instructions to the python script to generate all static files to that folder.~~

Final thoughts

I know I am, kind of, reinventing the wheel. But it’s somewhat satisfying to see exactly what I expect my page to have: Html files, a single css based on the script to generate the site. Nothing really fancy but meets my (very, very simple) needs. I think it can work for some time.

email-reporter

cesar@cesarkina.com (Cesar Kina) — Fri, 04 Dec 2020 00:00:00 +0000

I developed a java application that reads emails, parses the information and stores it into a data source. It’s email-reporter.

Currently, it implements:

Gmail: Queries emails by a given label and dates
Google Sheets: Reads and writes entries of what is parsed in the emails
Spring Task Scheduler: Ability to schedule a task to read emails and generate reports by a given cron setup

I wanted to have a sort of plain data of some services I usually consume and get receipts via email. E.g. Uber. I couldn’t find an easy way to export my trips’ data like the costs, discounts, and all the other fields that I saw interesting to keep track and that I have available in those email messages. Plus, I thought it could be fun to code something like this, and it was.

XPath selector on HTML with Java

cesar@cesarkina.com (Cesar Kina) — Wed, 30 Sep 2020 00:00:00 +0000

I wanted to explore how easy is to extract elements from HTML files or directly from website URLs programatically using Java. I found a third party library that gave me the desired results without much difficulties.

It’s XSoup.

Github Repo: https://github.com/code4craft/xsoup
Maven Repo: https://mvnrepository.com/artifact/us.codecraft/xsoup

With XPath expressions it is able to select the elements within the HTML using Jsoup as HTML parser.

In the example below, it extracts the posts of this website using the URL (https://cesarkina.com) , or reading the html of that exact url but downloaded into the repository.

The code

Repo: https://github.com/ckinan/learning/tree/main/java/parse-html

We got:

One single file App.java to place our logic
One single dependency in our pom.xml file: XSoup

App.java

I tried to explain the code with the docstrings and comments there to point out what the lines and methods are supposed to do.

package com.ckinan;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import us.codecraft.xsoup.Xsoup;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
/**
* Code snippet to find elements within a html (from file and url) using XPath.
*
* It uses a third party library called XSoup. Links:
* - mvn repository: https://mvnrepository.com/artifact/us.codecraft/xsoup
* - github repository: https://github.com/code4craft/xsoup
*
* This example reads a list of blog posts using a fixed XPath expression and prints the results in the console.
* */
public class App {
public static final String XPATH_BLOG_POST = "//div[contains(@class, 'leading-snug')]";
/**
* The method main of the program. It starts and orchestrates the process.
* It gets instances of Documents from specific sources and sends them to read specific elements
* by xpath expression.
*
* It retrieve documents from:
* - Files (by specifying the full path)
* - URL (of the website to be parsed)
*
* @param args
* Arguments of our application. Won't be used at all
* @throws java.io.IOException
*/
public static void main(String[] args) throws IOException {
// Calling the static methods that will get an instance of the HTML document and then evaluate
// the XPath expression to get a list of Blog Posts. First with a HTML file, then do the same with the website
// URL directly.
Document doc1 = App.getDocumentFromFile("src/main/resources/blog.html");
List<String> postsFromFile = App.readDocument(doc1, App.XPATH_BLOG_POST);
System.out.println(postsFromFile);
Document doc2 = App.getDocumentFromUrl("https://cesarkina.com/blog");
List<String> postsFromUrl = App.readDocument(doc2, App.XPATH_BLOG_POST);
System.out.println(postsFromUrl);
}
/**
* Read the content file of the given full path using "collect" from Stream Java 8+.
*
* Refs:
* - https://docs.oracle.com/javase/8/docs/api/java/nio/file/Paths.html#get-java.lang.String-java.lang.String...-
* - https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#lines-java.nio.file.Path-
* - https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.stream.Collector-
*
* Then it uses Jsoup to parse the content file into a Document instance.
*
* @param fullPath
* The full path of the file to be read and parsed using Jsoup.
* @throws java.io.IOException
*
* @return The parsed document which contains all the elements of the HTML file
*/
public static Document getDocumentFromFile(String fullPath) throws IOException {
System.out.println("Reading document from file: " + fullPath);
// Read the content file using Java 8+ Streams.
String html = Files.lines(Paths.get(fullPath)).collect(Collectors.joining(System.lineSeparator()));
return Jsoup.parse(html);
}
/**
* Read the content of the given website url using Jsoup
*
* @param url
* The URL of the website HTML to be read and parsed using Jsoup.
*
* @throws java.io.IOException
*
* @return The parsed document which contains all the elements of the HTML file
*/
public static Document getDocumentFromUrl(String url) throws IOException {
System.out.println("Reading document from url: " + url);
return Jsoup.connect(url).get();
}
/**
* Evaluates the given XPath expression on the given Document which is limited to assume that it's a list of
* Elements right now. Returns results of the XPath evaluation.
*
* @param doc
* The URL of the website HTML to be read and parsed using Jsoup.
*
* @param xpath
* The URL of the website HTML to be read and parsed using Jsoup.
*
* @throws java.io.IOException
*
* @return The list of string values resulted from the XPath expression.
*/
public static List<String> readDocument(Document doc, String xpath) {
List<String> result = new ArrayList<>();
// It first extract the Element instances of the html content using the given Document and the XPath expression
// Note: Xsoup uses Jsoup as the HTML parser. Xsoup "evaluates" a Document which is an instance variable created
// by Jsoup
List<Element> elements =
Xsoup.compile(xpath).evaluate(doc).getElements();
for(Element e: elements) {
// Each Element object has multiple properties available to be extracted, for example the HTML tag name.
// For this code snipped, we are interested into extract the text of the element.
// Example: <p>Hello world</p> -> e.text() would return "Hello world"
result.add(e.text());
}
return result;
}
}

pom.xml (dependences section only)

<dependencies>
<!-- https://mvnrepository.com/artifact/us.codecraft/xsoup -->
<dependency>
<groupId>us.codecraft</groupId>
<artifactId>xsoup</artifactId>
<version>0.3.1</version>
</dependency>
</dependencies>

Testing

Run the App.java class, and it should get this printed in the Console:

Reading document from file: src/main/resources/blog.html
[Sat 22:Remote Debug Spring Boot App in a Docker Container using IntelliJ IDEA, Thu 13:Cheat Sheet - Git Commands, Mon 20:Spring Security - Credentials from JSON request, Fri 03:Spring Security - Filter Chain, Sun 28:Spring Security - Form Login & Logout, Wed 17:KnockoutJS - Register Component, Mon 01:Spring Security - Getting Started, Sun 17:Tailwind CSS - First Steps, Fri 24:Project Github Pull Request Viewer, Fri 13:Github API - OAuth tokens for apps, Sat 07:Netlify Functions, Fri 07:Github API - First Steps, Sun 05:Auth0 - Simple Example with Vanilla JS, Tue 08:Intro]
Reading document from url: https://cesarkina.com/blog
[Sat 22:Remote Debug Spring Boot App in a Docker Container using IntelliJ IDEA, Thu 13:Cheat Sheet - Git Commands, Mon 20:Spring Security - Credentials from JSON request, Fri 03:Spring Security - Filter Chain, Sun 28:Spring Security - Form Login & Logout, Wed 17:KnockoutJS - Register Component, Mon 01:Spring Security - Getting Started, Sun 17:Tailwind CSS - First Steps, Fri 24:Project Github Pull Request Viewer, Fri 13:Github API - OAuth tokens for apps, Sat 07:Netlify Functions, Fri 07:Github API - First Steps, Sun 05:Auth0 - Simple Example with Vanilla JS, Tue 08:Intro]

Tooling

Something uselful to “test” the XPath expression to see if it’s a valid one or if will work, it is this chrome extension: XPath Helper. For the above example I introduced the XPath expression //div[contains(@class, 'leading-snug')] and ensure the results made sense.

Final thoughts

I needed an XPath selector, and Xsoup worked well and without a lot of boilerplate, just like what I was looking for. So thanks Xsoup!

Remote Debug Spring Boot App in a Docker Container using IntelliJ IDEA

cesar@cesarkina.com (Cesar Kina) — Sat, 22 Aug 2020 00:00:00 +0000

I had to setup a local environment for a Spring Boot application running in a Docker container. Naturally, I needed to find a way to debug my app, which is the step I want to explain here.

Prerequisites

IntelliJ IDEA
Docker

Scope

We will have only one (GET) endpoint that receives a “value” as string parameter and return that back to the client.
We want to put a breakpoint in the controller to demonstrate the debug configuration is working properly.
We don’t want more logic in our code as the goal is to get our debug config ready in this local/docker/springboot environment.

Create the project

Generate and Download a Spring Boot project with only “Spring Web” as a dependency.

Open the project in IntelliJ.

Create the resource controller

Open the DemoApplication class and create a resource controller, in this example mapped with alias /echo, that is simply going to return the value from the request.

package com.ckinan.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
@SpringBootApplication
public class DemoApplication {
@GetMapping("/echo")
public String resource(@RequestParam(value = "value", defaultValue = "insert a 'value' query param") String value) {
return "echo: " + value;
}
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}

Package your app

Execute the following command manually to get the jar file of your project generated using maven.

./mvnw package

You should have the jar file created in the ./target folder. In this example, it should be under the name ./target/demo-0.0.1-SNAPSHOT.jar

Create the Dockerfile

Create a Dockerfile in the root of the project.

FROM openjdk:14-jdk-alpine
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

We are giving the instruction to Docker to build an image from openjdk:14-jdk-alpine and copy our recently created jar file that will be used in our container.

Build the image

Run the docker build command to create the Docker Image.

docker build -t ckina/demo-spring-boot-docker-debug .

The image will be created with the repository name ckina/demo-spring-boot-docker-debug, change that if you want.

Remote debug your container

Choose one of the following ways to run your docker container: with docker run or docker-compose.

With docker run

Execute the docker run command using the Docker Image we’ve just created.

docker run --name demo-spring-boot-docker-debug -e "JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005" -p 8080:8080 -p 5005:5005 ckina/demo-spring-boot-docker-debug

With docker-compose

Create a docker-compose.yml file and execute docker-compose.

version: '3.3'
services:
spring-boot:
container_name: demo-spring-boot-docker-debug
build: .
ports:
- '8080:8080'
- '5005:5005'
environment:
- JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005

Options:

--name: We are naming our container as demo-spring-boot-docker-debug, which can be changed as needed.
-p 8080:8080: Expose our Spring Boot application outside the docker container
-p 5005:5005: Expose the port that will be used to connect our IDE with the application to do the remote debug
-e: Set environment variables, in this case, we want to use the -agentlib:jdwp option for debugging. (We will see more details of it later)

Run:

docker-compose up --build

Verify your endpoint is working by opening this in a browser: http://localhost:8080/echo?value=foo

It should give you echo: foo in the response of the call.

Create the remote debug configuration in IntelliJ and attach the debugger

Go to Edit Configurations and create a Remote config:

Note: It’s important to have the command line arguments matching with our configuration we introduced into our docker container. In this case this is what should match:

-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005

Go back to your IDE and do the following:

Set a breakpoint in your controller, right in the line where the string value is returned
Click on the debug button
You should have your IDE connected as the debugger of your Spring Boot application

Now execute one more time http://localhost:8080/echo?value=foo

The breakpoint in the IDE should be activated. We are done.

Details on the command line argument for the Remote Debug

Here we have the details about the -agentlib:jdwp option we introduced in the environment variables of our docker container:

Format: -agentlib:jdwp=<name1>[=<value1>],<name2>[=<value2>]...
transport=dt_socket: The way you want to connect the debug. It can be done through Socket Transport or Shared Memory Transport. In this example we’re using the Socket Transport.
server=y: We want to have our application listening to the debugger to be attached, in this case the debugger will be the IDE.
suspend=n: We don’t want to wait until the debugger is attacher to completely start the Spring Boot application.
address=*:5005: Listen for a socket connection on port 5005, which is the port number we want to configure in our IDE.

More details about the -agentlib:jdwp option: https://docs.oracle.com/en/java/javase/14/docs/specs/jpda/conninv.html

Cheat Sheet - Git Commands

cesar@cesarkina.com (Cesar Kina) — Thu, 13 Aug 2020 00:00:00 +0000

My Cheat Sheet for: Git Commands.

Gathering quick references I used to search over and over again. Good to have them handy and supposed to (slowly) grow.

git log

Git log with some styling (colors and specific order I like to see)

$ git log --pretty=format:"%C(Green)%h %C(Red)[.%D.] %C(Blue)[%an] %C(Yellow)[%s] %C(Magenta)[%ad (%ar)]"

Git log, raw info, tabular style

$ git log --pretty=format:"%h | %ad | %><(23,trunc)%ar | %><(15,trunc)%an | %<(72,trunc)%s | %D" --date=format:"%a %Y-%m-%d %H:%M:%S"

git diff

Get number of files changed, lines inserted and lines removed between two branches or commits

$ git diff --shortstat master..develop
13 files changed, 90 insertions(+), 410 deletions(-)
$ git diff --shortstat 499a153..8059ee2
3 files changed, 69 insertions(+), 1 deletion(-)

Get diff between two branches or commits with only relevant information (not the whole diff)

# File names only
$ git diff --name-only origin/main..develop
blog/intro/index.mdx
generator/MarkdownRenderer.py
resources/404.html
resources/about.html
resources/index.html
resources/layout.html
resources/notes.html
resources/post.html
resources/styles.css
# File names with status (modified, deleted, added)
$ git diff --name-status origin/main..develop
M blog/intro/index.mdx
M generator/MarkdownRenderer.py
M resources/404.html
M resources/about.html
M resources/index.html
M resources/layout.html
M resources/notes.html
M resources/post.html
M resources/styles.css
# With stats
$ git diff --stat origin/main..develop
blog/intro/index.mdx | 19 +++++++++----------
generator/MarkdownRenderer.py | 9 +++------
resources/404.html | 2 +-
resources/about.html | 2 +-
resources/index.html | 2 +-
resources/layout.html | 12 +++++-------
resources/notes.html | 2 +-
resources/post.html | 3 +--
resources/styles.css | 36 ++++++++++--------------------------
9 files changed, 32 insertions(+), 55 deletions(-)

git stash

Stash your changes quickly

$ git stash

Stash your changes with a custom message:

$ git stash push -m "your message here"

See your stash entries

$ git stash list
stash@{0}: On main: your message here

See the diff of a given stash entry

$ git stash show -p stash@{0}
diff --git a/blog/cheat-sheet-git-commands/index.mdx b/blog/cheat-sheet-git-commands/index.mdx
index 0a038ac..6134c05 100644
--- a/blog/cheat-sheet-git-commands/index.mdx
+++ b/blog/cheat-sheet-git-commands/index.mdx
@@ -6,12 +6,16 @@ slug: cheat-sheet-git-commands
:
(more)

Save stash diff on disk

$ git stash show -p stash@{0} > ~/your_diff.patch

Apply stash to your current branch. Note: The stash will be removed from the stash store

$ git stash pop stash@{0}

Clean your stash entries

# Only one entry
$ git stash drop stash@{2}
# Delete all your stash entries
$ git stash clear

git reset

Unstage changes made on certain files. (Changes can be staged again with git add command)

git reset file1.py file2.py

GitWeb

To launch git web view

git instaweb --httpd=webrick

And stop with

git instaweb --httpd=webrick --stop

In browser: http://localhost:1234

Ref: https://git-scm.com/book/en/v2/Git-on-the-Server-GitWeb

Misc

Stage all changed files even the ones living in parent directories of a git repo

git add -A
# or...
git add --all

Spring Security - Credentials from JSON request

cesar@cesarkina.com (Cesar Kina) — Mon, 20 Jul 2020 00:00:00 +0000

One more example of how flexible Spring Security is. This time, we will take a look at how we can override the default behavior to read credentials from any requests to start working with JSON format and authenticate users based on that.

Introduction

Once you include the Spring Security dependency into your project, it gives you a lot of features out of the box. A pre-defined login form is one of them, for any protected resource without an authenticated user, the application would automatically redirect you to a Login Page (a form). Any form submission is sent to the endpoint /login (method POST) and the filter chain of Spring Security takes action. It extracts the username and password parameters from the request and use them to perform the authentication process.

That is the default behavior coming from Spring Security. In the class UsernamePasswordAuthenticationFilter, it has a method called attemptAuthentication, which receives two parameters: a request and a response (HttpServletRequest and HttpServletResponse). This class is part of the filter chain (by default), and if there is a login form submission, then the request will pass through there.

But what if we don’t want to send the credentials in a form submission, and our use case requires an endpoint that allows clients to authenticate users by sending their credentials within a JSON body. In that case, the default filter won’t support that format. The filter will look at the request-parameters and will try to find the username and the password params names. In fact, it is very explicit in the docstrings of the UsernamePasswordAuthenticationFilter class:

Login forms must present two parameters to this filter: a username and password.

Here I will explain one way to get credentials from a JSON request and continue with the rest of the security filter chain.

Scope

Create a custom filter to read the credentials from the request in a JSON format and include it in the security filter chain in replacement of the UsernamePasswordAuthenticationFilter
No UI. Everything will be tested via Postman
Set users details in memory

Pre-requisites

Java 8 or higher
IDE of your preference (Intellij Idea, Eclipse, etc)

Create project

Go to http://start.spring.io and generate a new project. These are the key pieces for this code example:

Project: Gradle
Language: Java
Dependencies: Spring Web, Spring Security

You can use this direct link with all the configuration already set: link

Create the components

In the project we’ve just downloaded, let’s create the following classes under the main package (com.ckinan in my case):

Application: It comes from the zip file we’ve just downloaded. It will start the Spring Boot application for us. No changes required here
Controller: Two “resources” will be exposed in this controller: A protected resource and a public resource (names are self-explanatory)
SecurityConfiguration: Contains our security configuration and we will provide the in-memory user to the AuthenticationManager by using its configurer
CustomFilter: Replacement of UsernamePasswordAuthenticationFilter in the filter chain. It will read the credentials from the JSON payload coming within the HttpServletRequest

Project Structure

Review the components

Controller

package com.ckinan;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class Controller {
@RequestMapping("/protected-resource")
public String protectedResource() {
return "The Protected Resource";
}
@RequestMapping("/public-resource")
public String publicResource() {
return "The Public Resource";
}
}

Includes two endpoints (both GET method) that basically return some string values:

/protected-resource: This endpoint will be protected by our Spring Security configuration. By “protected”, it means clients need to authenticate first to access the resource.
/public-resource: Authentication is not needed to access this resource.

Note: Later we will define the security restrictions for both endpoints in SecurityConfiguration class.

CustomFilter

package com.ckinan;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.security.authentication.AuthenticationServiceException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter;
import org.springframework.security.web.util.matcher.AntPathRequestMatcher;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Map;
public class CustomFilter extends AbstractAuthenticationProcessingFilter {
protected CustomFilter() {
super(new AntPathRequestMatcher("/login", "POST"));
}
@Override
public Authentication attemptAuthentication(
HttpServletRequest request, HttpServletResponse response) throws AuthenticationException {
String username, password;
try {
Map<String, String> requestMap = new ObjectMapper().readValue(request.getInputStream(), Map.class);
username = requestMap.get("username");
password = requestMap.get("password");
} catch (IOException e) {
throw new AuthenticationServiceException(e.getMessage(), e);
}
UsernamePasswordAuthenticationToken authRequest = new UsernamePasswordAuthenticationToken(
username, password);
return this.getAuthenticationManager().authenticate(authRequest);
}
}

This filter is the one in charge of reading the credentials from the request in JSON format. It will act as replacement of the UsernamePasswordAuthenticationFilter, which we are removing from our spring security configuration in favor of the new CustomFilter.

The constructor is passing an AntPathRequestMatcher object with two params:

pattern: Any request coming from the /login path will be considered by this filter to perform the authentication process. It could be anything that matches your needs as API pattern or path, for example: /api/login or /sign-in or whatever
httpMethod: We want to process any request as POST method

We are also overriding the method attemptAuthentication, which is going to extract the username and password from the request JSON payload. There are certainly multiple ways to extract the JSON values and assign them into java objects, but for now, I’m using ObjectMapper and assigning the values to a Map. Later they will be assigned to their corresponding String variables.

Once we get the credentials, we need to perform the authenticate process, which is provided by the AuthenticationManager. Quoting from the docs https://spring.io/guides/topicals/spring-security-architecture :

Spring Boot provides a default global AuthenticationManager (with just one user) unless you pre-empt it by providing your own bean of type AuthenticationManager. The default is secure enough on its own for you not to have to worry about it much, unless you actively need a custom global AuthenticationManager. If you do any configuration that builds an AuthenticationManager you can often do it locally to the resources that you are protecting and not worry about the global default.

In this example, we are configuring our in-memory user (see later in our SecurityConfiguration class) into the AuthenticationManager. So any username and password from a JSON payload will be verified against the in-memory user we have there. I think this is the meat of this example.

For that to happen, the method authenticate of the AuthenticationManager receives a token, which in this case we are initializing a UsernamePasswordAuthenticationToken with the credetials coming from the request.

One important note on this, if you visit the UsernamePasswordAuthenticationFilter class (from the codebase of Spring Security), you will notice that I’m trying to mimic the logic in its attemptAuthentication method, which by default it gets the credentials coming as form parameters (link)

SecurityConfiguration

package com.ckinan;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;
@EnableWebSecurity
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
@Override
protected void configure(AuthenticationManagerBuilder auth) throws Exception {
auth.inMemoryAuthentication().withUser("cesar").password("{noop}s3cr3t").roles("USER");
}
@Override
protected void configure(HttpSecurity http) throws Exception {
CustomFilter mupaf = new CustomFilter();
mupaf.setAuthenticationManager(authenticationManager());
http
.csrf().disable()
.addFilterAt(
mupaf,
UsernamePasswordAuthenticationFilter.class)
.authorizeRequests()
.antMatchers(HttpMethod.GET, "/protected-resource").authenticated()
.antMatchers(HttpMethod.POST, "/login").permitAll();
}
}

For simplicity, here we are working with only one user loaded from memory:

Username: cesar
Password: s3cr3t

The {noop} means we are going to use the NoOpPasswordEncoder password encoder, which simply does not do anything to the password we are providing s3cr3t (it won’t get encoded). This is not secure obviously, but will work for this example.

The AuthenticationManagerBuilder will allow us to configure the above in-memory user into the AuthenticationManager. So that any request that passes through the CustomFilter, will try to “authenticate” the incoming credentials against this user details manager.

Creates new instance of the CustomFilter, in which we want to provide the authenticationManager we should already have configured at this point. Then insert the filter in the same position as the UsernamePasswordAuthenticationFilter would have been.

Then, configure the antMatchers for the two resources (one public one protected).

Note: I need another space to explore the csrf configuration. For now, we need to disable the feature, otherwise requests won’t get until the CustomFilter within the filter chain

Run and test

Run the application with the following instruction in terminal (need to cd to the project directory):

$ ./gradlew bootRun

Testing with Postman, it will fail trying to access the protected-resource because the user is not authenticated (yet)

Use the /login endpoint to authenticate the user. The Not Found error from the response refers to the / resource which we simply don’t have in our controller. Remember we only have set two endpoints: protected-resource and public-resource.

Now, try to access the protected-resource once again. This time, the user should be already authenticated.

At any point of time, with or without a successful authentication, the public-resource should be accessible.

Final thoughts

During this example, I noticed there is another area to explore: how AuthenticationManager, ProviderManager and AuthenticationProvider are connected all together to allow the application setup a custom authentication process.
Nothing prevents us to customize the way we extract credentials from the requests. And one more time, it proves the flexibility of Spring Security to do some work like that. All I did was to mimic what UsernamePasswordAuthenticationFilter does to capture the username and password from a HttpServletRequest object.
It’s pretty important to be aware of the filters that Spring Security provides. There is a comprehensive list of filters in this link. So if there is another behavior we want to customize in regards of what is happening in the filter chain, then we need to know where exactly (and how) that should happen. A breakpoint in FilterChainProxy is a good point to start.

Spring Security - Filter Chain

cesar@cesarkina.com (Cesar Kina) — Fri, 03 Jul 2020 00:00:00 +0000

After playing with few basic configurations of Spring Security, I noticed there is something important to understand before moving to check other features in the framework, which is its Filter Chain. This time I’ll take a look at how Spring Security acts on the requests coming from the client, which involves a set of Filters, each of them with a very specific goal.

Note: This is not a tutorial, this is just some notes of what I understand (as of today) about Spring Security and its Filter Chain.

Introduction

The most complete explanation I could find regarding how the Security Filter Chain is built in the entire architecture is in the official documentation in here: https://docs.spring.io/spring-security/site/docs/5.3.2.RELEASE/reference/html5/#servlet-architecture

I’ll write down what I interpret from what I already interacted with the module in combination with what the documentation is trying to explain.

Main Components

There are five pieces (in the documentation I pointed out in the previous section) that I want to visit here. They are:

Filters

First of all, “Filters” (in the Servlet’s world) came from the Java Specification Requests (JSR) #53. Look at this:

Ref: https://jcp.org/en/jsr/detail?id=53
Direct link to the PDF (See Page #43): https://download.oracle.com/otn-pub/jcp/7840-servlet-2.3-spec-oth-JSpec/servlet-2_3-fcs-spec.pdf?AuthParam=1593651764_bd0a8306800ffcc92df386c00a6c89f0

Filters are some pieces of code that do something before or after a Request gets processed by a Servlet. The web container (e.g. Tomcat) will know that.

Filters have multiple purposes, it is not only restricted to be for Security mechanisms. They are meant to read or write (any logic in general) with the Request. For example: one filter can print some audit logs, another one can add some headers to the request, and so on.

In the Spring Security’s world, we have a set of Filters strategically ordered to prevent subsequent filters or the controller itself being called if they find the request not matching with the security configuration.

The most straightforward example may be, imagine a user entering a wrong password, there is one filter dedicated to verify whether the password is correct or not. If not, then the at some point of the Filter Chain, one of the Filters will reject the request and respond with a 401 status code to the client.

DelegatingFilterProxy

DelegatingFilterProxy is a class registered as Standard Filter in the Container, but the container doesn’t know anything about Spring Beans at this point. DelegatingFilterProxy is the support to establish communication between the Standard Filters and the Spring Application Context.

Notice that I didn’t mention anything about Spring Security yet, that’s because DelegatingFilterProxy is a class located in Spring Web and not Spring Security. Which is interesting because from there I interpret that this proxy can be used by something else not necessarily related to Spring Security.

FilterChainProxy

FilterChainProxy is a filter located in Spring Security module.

It takes a list of filters and creates something called VirtualFilterChain (a private class within FilterChainProxy), which is going to take the list of the Security Filters and start the chain.

I want to point this out that seems to be pretty useful, quoting Spring Security docs:

First, it provides a starting point for all of Spring Security’s Servlet support. For that reason, if you are attempting to troubleshoot Spring Security’s Servlet support, adding a debug point in FilterChainProxy is a great place to start.

SecurityFilterChain

SecurityFilterChain contains the list of all the filters involved in Spring Security. This interface expose a method List<Filter> getFilters() that returns all the filters such as the UsernamePasswordAuthenticationFilter or LogoutFilter.

In Spring Security, one or more SecurityFilterChains can be registered in the FilterChainProxy.

The following picture illustrates all the components I tried to explain so far:

SecurityFilterChain, by Spring Security Docs

Security Filters

Finally, at the end of all this hierarchy (Standard Filter > DelegatingFilterProxy > FilterChainProxy > SecurityFilterChain), there is list of possible security filters that we can configure in our applications. I’ll just mention the ones that I kind of interated so far:

LogoutFilter
UsernamePasswordAuthenticationFilter
DefaultLoginPageGeneratingFilter
BasicAuthenticationFilter
SessionManagementFilter

In fact, everytime your Spring Boot application starts up, the list of filters are printed in the logs by default (in their corresponding order of execution). Example:

// Raw logs at startup time
// List of filters are printed there
2020-07-02 21:34:55.896 INFO 51095 --- [ main] o.s.s.web.DefaultSecurityFilterChain : Creating filter chain: any request, [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@7fe82967, org.springframework.security.web.context.SecurityContextPersistenceFilter@9596ce8, org.springframework.security.web.header.HeaderWriterFilter@26f46fa6, org.springframework.security.web.csrf.CsrfFilter@558756be, org.springframework.security.web.authentication.logout.LogoutFilter@80bfdc6, org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@6edcad64, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@75ae4a1f, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@476fe690, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@50850539, org.springframework.security.web.session.SessionManagementFilter@227a47, org.springframework.security.web.access.ExceptionTranslationFilter@3968bc60, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@2924f1d8]
// And this is the list of filters in a more readable way:
[
1. org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@5fa23c
2. org.springframework.security.web.context.SecurityContextPersistenceFilter@73c9e8e8
3. org.springframework.security.web.header.HeaderWriterFilter@59f93db8
4. org.springframework.security.web.csrf.CsrfFilter@13d9261f
5. org.springframework.security.web.authentication.logout.LogoutFilter@b606cb6
6. org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@2a3194c6
7. org.springframework.security.web.savedrequest.RequestCacheAwareFilter@de8039f
8. org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@4b790d86
9. org.springframework.security.web.authentication.AnonymousAuthenticationFilter@558756be
10. org.springframework.security.web.session.SessionManagementFilter@1aabf50d
11. org.springframework.security.web.access.ExceptionTranslationFilter@2e8a1ab4
12. org.springframework.security.web.access.intercept.FilterSecurityInterceptor@51c959a4
]

So, any requests coming from the client will pass through the filter chain and in that order.

Final thoughts

I started to write this article thinking of the Filter Chain as something strictily related to Spring Security. BUT, during all my reading I got that this part of the architecture in regards of how Spring Security Filters are added into the actual “standard” filter chain as something that you first need to know other “external” concepts about Filters, Servlets and the interfaces that Spring Web provides. Only then, all the illustrations and explanation in the official docs started to make sense to me.
Debugging one of my few simple examples during startup time helped me a lot to understand some pieces, specifically having some breakpoints in FilterChainProxy then going all the way down and all the way up in the whole call hierarchy was pretty useful. Still, there are A LOT of areas in the codebase of Spring Security and Spring Web that I don’t understand right now..
I’m impressed the quality of the docstrings Spring has in its codebase, I found a lot of insights there, like examples, code snippets, explanation in details (more than I could find in the official docs).

Refs

This is where most part of this post comes from: https://docs.spring.io/spring-security/site/docs/5.3.2.RELEASE/reference/html5/#servlet-architecture
Architecture Deep Dive in Spring Security - Joe Grandja @ Spring I/O 2017: https://www.youtube.com/watch?v=8rnOsF3RVQc
SpringOne 2GX 2012 - Getting Started with Spring Security 3.1 - Rob Winch: https://www.youtube.com/watch?v=k32KqrckLEE

Spring Security - Form Login & Logout

cesar@cesarkina.com (Cesar Kina) — Sun, 28 Jun 2020 00:00:00 +0000

I am still in the process of learning more about Spring Security. This time I created a simple example that involves Form Login and Logout customizations.

Scope

Happy path: Public page -> Login Page -> Protected resource
Non-happy path: Public page -> Login Page -> Login Error Page
Ability to logout once the user is authenticated (it’s on the Protected resource)
Customize the urls for login and logout that come by default from Spring Security
Create public and protected resources. This time, let’s do some html (server side rendered)
Create custom view to show whenever a login attempt fails
Focus only in these two configurers: Form Login & Logout

Out of scope

Do not create custom users. Let’s keep using the default user and password generated by Spring Security to avoid other distractions
Do not concentrate in file/project structure, let’s keep it simple in one single file

Pre-requisites

Java 8 or higher
IDE of your preference (Intellij Idea, Eclipse, etc)

Create project

Go to https://start.spring.io/ and add the following dependencies:

Spring Web
Spring Security
Thymeleaf (we will create some html templates to render the views in server-side)

Generate, download and open the project in your IDE. I’ll use Intellij IDEA.

Create the components

In the project that we just downloaded, let’s edit the SpringSecurity02Application.java file, which is our SpringBootApplication:

Create a class SecurityConfiguration, which extends WebSecurityConfigurerAdapter. We will override the configure(HttpSecurity) method to setup our login and logout processes
Create a class Controller, which will provide the endpoints for our views (login, logout, protected and public resources)

Then, under the resources folder of our application, in this case spring-security-02/src/main/resources/templates, let’s create the four templates we want to render in our flow:

login.html
login-error.html
protected-resource.html
public-resource.html

The project structure should look like:

The above is all what we will create (in terms of files and classes). Now let’s move to the details.

This is how our code should look like for the whole example (repo available on Github):

Review the components

From the previous picture, let’s take a look at each component and what each piece means.

Block #1. The Form Login Setup

formLogin(): Let Spring Security know we want to support authentication through a login form
loginPage("/custom-login"): Set the custom login url to GET the login form rendered and also to submit (POST) the form
usernameParameter("myUser"): Specify the parameter name to get the username from the request
passwordParameter("myPass"): Same than previous but for the password
defaultSuccessUrl("/protected-resource"): URL that we want to be redirected once it gets a successful login attempt
failureHandler(...): Provide a handler for any failed requests. This time, we want to configure a SimpleUrlAuthenticationFailureHandler, which is going to redirect the user to the given url in the constructor, in this case it’s /login-error

Block #2. The Logout Setup

logout(): Allow our application to customize the login “default” setup
logoutUrl("/custom-logout"): URL that will handle the logout process
logoutSuccessUrl("/"): Similar to defaultSuccessUrl, but for the logout process. The url that we want to be redirected once it gets a successful logout attempt

Block #3. The controller

The controller has four request handlers, all of them provide server-rendered views that will serve to our example to have the login authentication flow. Most of these views will work with the template engine Thymeleaf to access data from the application’s context.

/custom-login: Serve the login.html view:

spring-security-02/src/main/resources/templates/login.html
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:th="https://www.thymeleaf.org">
<head>
<title>Login Page</title>
</head>
<body>
<h1>Login Page</h1>
<form th:action="@{/custom-login}" method="post">
<div>
Username: <input type="text" name="myUser"/>
</div>
<div>
Password: <input type="password" name="myPass"/>
</div>
<div>
<input type="submit" value="Log in" />
</div>
</form>
</body>
</html>

In this template:

Action /custom-login will send the content (POST) from the client (the browser) to the server (our application)
Username and password <input>’s names should match with the usernameParameter(...) and passwordParameter(...) we specified to the formLogin() setup

/login-error: Serve the login-error.html view:

spring-security-02/src/main/resources/templates/login-error.html
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:th="https://www.thymeleaf.org">
<html>
<head>
<title>Login Error Page</title>
</head>
<body>
<h1>Login Error Page</h1>
<p>Reason: <span th:text="${session.SPRING_SECURITY_LAST_EXCEPTION.message}"></span></p>
<p>Go to <a href="/custom-login">Login Page</a></p>
</body>
</html>

In this template:

Print the error message for any failed login attempt. The error message is coming from an exception thrown by Spring Security and added into the session attribute: SPRING_SECURITY_LAST_EXCEPTION
A link that redirects to the login page

Note: Usually, we don’t need a separate page just to inform that provided credentials are wrong. But I wanted to create this one for the sake of exploring this feature…

/protected-resource: Serve the protected-resource.html view:

spring-security-02/src/main/resources/templates/protected-resource.html
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:th="https://www.thymeleaf.org">
<head>
<title>Protected Resource</title>
</head>
<body>
<h1>Protected Resource</h1>
<p>Hey, I am a protected resource. Only authenticated users can see me.</p>
<form th:action="@{/custom-logout}" method="post">
<div>
<input type="submit" value="Logout" />
</div>
</form>
</body>
</html>

In this template:

Points to the action /custom-logout. It should match with what we set in logoutUrl(...) during the logout() setup.
Notice the method for this form is POST. According to the documentation:

If CSRF protection is enabled (default), then the request must also be a POST. This means that by default POST “/logout” is required to trigger a log out. If CSRF protection is disabled, then any HTTP method is allowed.

In our example, we didn’t disabled the CSRF protection, so we want to do POST. More about CSRF: https://en.wikipedia.org/wiki/Cross-site_request_forgery

/: Serve the public-resource.html view:

<html>
<head>
<title>Public Resource</title>
</head>
<body>
<h1>Public Resource</h1>
<p>Hey, I am a public resource. Anyone can see me.</p>
<p>Login to see protected resources. <a href="/custom-login">Login Page</a></p>
</body>
</html>

This view can be accessed without being authenticated. Remember that we only require authentication for the protected-resource, which we defined in our SecurityConfiguration class:

 .authorizeRequests()
.antMatchers(HttpMethod.GET, "/protected-resource").authenticated();

Run and test

Run the application with the following instruction in terminal (need to cd to the project directory):

$ ./gradlew bootRun

Grab the auto-generated password from logs and wait until Spring Boot is fully initialized:

Using generated security password: 0c6fc7f8-34b1-4c04-a0f6-dbaa8eb8c21f

Open http://localhost:8080 in a browser

This is the public resource (no need to authenticate so far)

Click on the “Login Page” link and the Login Form should be rendered. Enter user as the username (defaulted by Spring Security) and the auto-generated password. Then click on “Log in” button

The protected resource should be displayed

If during login form credentials are invalid, then login-error view should be displayed

If user is not authenticated and tries to access protected-resource through the browser via http://localhost:8080/protected-resource , then the login page should be displayed. We didn’t do anything but just telling Spring Security that resource needs authentication, then Spring Security is responsible of redirect the request to the Login Page

Final Thoughts

From this example, I learned the following:

The more I read about Spring Security, the more flexibility I find in there, which is awesome. The problem is that I (personally) learn by doing small things in the most isolated way. That is why I approached this post to only elaborate on Login and Logout very specific configurations. There are more features in those configurers that I didn’t take, but at least this example helped me to know what to see, what to expect and where to look for help, which is something for the next point.
Some pieces of code in this example where just copied from the Spring Security docstrings. For example:

https://docs.spring.io/spring-security/site/docs/current/api/org/springframework/security/config/annotation/web/configurers/FormLoginConfigurer.html#loginPage-java.lang.String-

In the above link, I found a Example login.jsp, which gave me a clear idea of what I have to have in login.html. I think Spring Security team makes a great job documenting their code.

Found really useful navigate Spring Security codebase and check the classes hierarchies, method usages, stuff like that. I downloaded the spring-security repo and do that exercise any time I needed to look further
There are still some things I need to understand in the whole architecture of Spring Security. To me, how the request is redirected using the handlers is sort of mistery. I’d like to see how the filter chain works under the hood, and several other concepts I usually see when I google for something. I guess I’ll figure them out when the time comes.

Refs

Github Repo: https://github.com/ckinan/learning/tree/main/java/spring-security-02
Spring Security - Docs: https://docs.spring.io/spring-security/site/docs/current/api/overview-summary.html
Thymeleaf - Access data: https://www.thymeleaf.org/doc/articles/springmvcaccessdata.html

KnockoutJS - Register Component

cesar@cesarkina.com (Cesar Kina) — Wed, 17 Jun 2020 00:00:00 +0000

One more time, I convinced myself how important is sometimes get away from my codebase and isolate my tests to understand foundations. This time, got something with KnockoutJS and how components are registered.

Motivation

Recently I wanted to understand how a KnockoutJS components can be created. So, I went into the official docs and found good explanations about it:

https://knockoutjs.com/documentation/component-overview.html

Instead of trying things directly to the codebase I wanted to introduce new components, I first wanted to create really simple examples in an isolated way.

I created a codepen for this excercise, which helped me a lot to quickly make changes and play with the things I was learning during this.

https://codepen.io/ckinan/pen/eYJBjYR

Playground

The following component called my-component configures a viewModel that creates a new object that handles the states of the values in the form. Also, tells the components to be synchronously loaded and finally declares the template to be rendered in the DOM.

var MyViewModel = function(first, last) {
this.firstName = ko.observable('Cesar');
this.lastName = ko.observable('K');
this.fullName = ko.pureComputed(function() {
return this.firstName() + " " + this.lastName();
}, this);
};
ko.components.register('my-component', {
viewModel: {
createViewModel: function(params) {
return new MyViewModel(params);
},
},
synchronous: true,
template: `
<div>
<p>First name: <input data-bind="textInput: firstName" /></p>
<p>Last name: <input data-bind="textInput: lastName" /></p>
<strong>Hello, <span data-bind="text: fullName"> </span>!</strong>
</div>
`
});

I got 4 ways to use components (probably there are more variations). For now, this is what I got.

Approach #1

Bind the component by specifying its name directly.

<div data-bind='component: "my-component"'></div>

Approach #2

Bind the component by specifying an object with the name of it.

<div data-bind='component: { name: "my-component" }'></div>

Note: In that object, you can also pass the params to the component if there is any. This example only passes the name.

So far, approaches #1 and #2 are taken from here https://knockoutjs.com/documentation/component-binding.html

Approach #3

Using component as “Custom elements”. Ref: https://knockoutjs.com/documentation/component-custom-elements.html

<my-component></my-component>

Approach #4

Just like in Approach #2, but in Javascript having one component to inject into another component.

New component for this example is a divider. (Only a template, no view models this time)

ko.components.register('my-divider-component', {
template: `
<div style="border-top: 3px solid #bbb;"></div>
`
});

Then, create an object that will provide the information about the component (the name and optionally the params).

const MyDividerComponent = function() {
const self = this;
self.component = () => {
return {
name: 'my-divider-component'
}
};
};
const myDividerComponent = new MyDividerComponent();

Finally, inject this component into another component while creating the template. (I’ll use my-component example but will add one more line in the template)

ko.components.register('my-component', {
viewModel: {
createViewModel: function(params) {
return new MyViewModel(params);
},
},
synchronous: true,
template: `
<div>
<p>First name: <input data-bind="textInput: firstName" /></p>
<p>Last name: <input data-bind="textInput: lastName" /></p>
<strong>Hello, <span data-bind="text: fullName"> </span>!</strong>
<div data-bind="component: myDividerComponent.component()"></div>
</div>
`
});

That way, myDividerComponent.component() will actually return this object:

{
name: 'my-divider-component'
}

With that object, the template will inject the component just like I explained in approach #2.

Final thoughts

With this simple examples, I just want to point out two main things:

KnockoutJS has multiple ways to use a component. I’m just aware of the four ones in this article, which is good for now to understand some things that kind of overwhelmed me before doing these excercises.
I am finding it super useful when I create this “playground” things to understand something without all the distractions that a big codebase can have (not complaining, just my way to learn… maybe). Easy to share, easy to remember, easy to extend or copy.

Spring Security - Getting Started

cesar@cesarkina.com (Cesar Kina) — Mon, 01 Jun 2020 00:00:00 +0000

Getting started with Spring Security by doing a simple example to understand the basics.

Pre-requisites

Java 8 or higher
IDE of your preference (Intellij Idea, Eclipse, etc)

Create project

Go to https://start.spring.io/ and add the following dependencies:

Spring Web
Spring Security

Click on the GENERATE button. A zip file should be downloaded. Unzip it and we are ready to start.

Run the application

cd to the project directory, and execute the following:

$ ./mvnw spring-boot:run

Logs should look like:

Spring Security provides you a default password (for a default user) out of the box just by adding the Spring Security dependency. In the above screenshot, the password is c916dd57-3b15-49e8-906e-163c7aadf1b5 for the user user.
The filter chain created for your requests. There are 15 filters in the chain added for you and in the order they are printed in the logs within the array. If you open a text editor and put all them in one line each, you should see something like:

org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@f951a7f
org.springframework.security.web.context.SecurityContextPersistenceFilter@360bc645
org.springframework.security.web.header.HeaderWriterFilter@6cd5122d
org.springframework.security.web.csrf.CsrfFilter@3fe46690
org.springframework.security.web.authentication.logout.LogoutFilter@79fd6f95
org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@6726cc69
org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter@2241f05b
org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter@20cece0b
org.springframework.security.web.authentication.www.BasicAuthenticationFilter@5300cac
org.springframework.security.web.savedrequest.RequestCacheAwareFilter@5d51e129
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@109a2025
org.springframework.security.web.authentication.AnonymousAuthenticationFilter@4c777e7b
org.springframework.security.web.session.SessionManagementFilter@10c07b8d
org.springframework.security.web.access.ExceptionTranslationFilter@b16e202
org.springframework.security.web.access.intercept.FilterSecurityInterceptor@fd9ebde

Each of these have a purpose, and they make the whole flow of the security layer possible. Remember, they are filters chained that will take actions in that order whenever a new request comes.

The port that your application is running. In this case, it’s in 8080.

Create a rest controller
Secure a REST API

Open the application

Open http://localhost:8080 in your browser, a login page should be displayed:

By default and by just adding the Spring Security dependency to the project, it generates a login page that sends the credentials to a generated endpoint /login that is in charge of verify whether the username and password are valid for the application and give access to the protected resources.

Let’s enter the credentials printed in logs into the form and click on the Sign in button (or just press enter).

Username: user
Password: c916dd57-3b15-49e8-906e-163c7aadf1b5

An error page is displayed:

This is expected. In order to understand what happened, let’s go to the Network Tab in the Dev Tools.

The credentials were sent to the endpoint login. Spring Security created that for us for free and the request will be processed by the filter chain we saw in the logs earlier.
Information are sent from the pre-generated form. Notice the keys are already pre-defined username, password and a third one _csrf, which is meant to tackle the CSRF vulnerabilities. All them will be interpreted by the login endpoint.

So, why we get an error page if I utilized the correct credentials? Let’s see the second record in the Network Tab.

The resource that was asked was: http://localhost:8080/. Think of it as it requested the / resource from http://localhost:8080.
404 (not found). It means, the resource / doesn’t exist.
The referer or who had initiated that request was http://localhost:8080/login which is the location where the credentials were sent through the form.

The error message refers to a resource that was not found. And that is because Spring Security only provides us the security stuff for the resources we want to protect. But, as we didn’t touch anything in the codebase, actually we didn’t even open the project in the IDE, then there is nothing to protect (yet) and the resource / (I suspect) is the default path right after a successful login is attempted.

One more thing to notice, in Application Tab there in the Dev Tools, there is a new Cookie created under the name JSESSION.

That is what keeps the already-logged-in user in session in the Spring Security context.

Finally, there are still more particulars working behind the scenes, in the following link there is a list what is happening to Spring Boot once the Spring Security dependency is included into the project.

Link: https://docs.spring.io/spring-security/site/docs/5.3.2.RELEASE/reference/html5/#servlet-hello-auto-configuration

A lot of things have happened by just adding the Spring Security dependency into the project. I didn’t write any single line of code to get all the above stuff with Spring Security out of the box.

Now, let’s go to the codebase and some documentation to understand where all these came from.

The content of the project

The above is all what the project has. Actually, in https://start.spring.io/ , there is a button EXPLORE that you can see the content of the zip before downloading it. Well, as I didn’t touch a thing, that project tree should be exactly the same as I downloaded (except for the auto-generated stuff of my IDE).

There are 3 key files that I’d like to mention so far:

SpringSecurity01Application is the class that initiates the application. In fact, this is not a Spring Security thing but from Spring Boot.
mvnw, this is the script that we utilized to run our application with the command ./mvnw spring-boot:run. This is only one way to start the application, you can also run this by executing the generated jar after packaging the component with maven, maybe other techniques. For now, I’ll keep continue using this method.
pom.xml registered two important dependencies:

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>

The most relevant one is spring-boot-starter-security, which is giving us all the security features that we saw so far in this example. The other one spring-boot-starter-web is will enable the RESTful endpoints that are utilized in this example.

From here, there are a lot of features to incorporate into the project, and there are several ways to do the same thing. For now I’ll do a simple exercise that will allow me to explain couple of more things.

Securing a REST API

I’ll write a very simple REST API that will just return a string value and have Spring Security protecting the endpoint.

Create the Controller Class

Let’s create two endpoints. One protected and one public.

// ./Controller.java
package com.ckinan;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class Controller {
@RequestMapping("/protected-resource")
public String protectedResource() {
return "The Protected Resource";
}
@RequestMapping("/public-resource")
public String publicResource() {
return "The Public Resource";
}
}

In this controller, there is no evidence of security for any of these endpoints. The controller doesn’t even know they will be protected. They could, but we will have that in a separate class. For now, it’s important to know the following:

The endpoint /protected-resource will be protected by Spring Security with some basic instructions we will set later in a separate class.
The endpoint /public-resource will be publicly available. Meaning, no authentication will be required.

Create the Security Configuration Class

All the configuration will live in a separate class in this step.

// ./SecurityConfiguration.java
package com.ckinan;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
@EnableWebSecurity
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
@Override
protected void configure(HttpSecurity http) throws Exception {
http
.formLogin()
.and()
.authorizeRequests()
.antMatchers(HttpMethod.GET, "/protected-resource").authenticated()
.antMatchers(HttpMethod.GET, "/public-resource").permitAll();
}
}

In this configuration class we define the security features we want to add to our endpoints. Let’s see some key points:

@EnableWebSecurity: This annotation will set effectively the Spring Security configuration we define in the class. It works together with the class we are extending: WebSecurityConfigurerAdapter.
WebSecurityConfigurerAdapter: When we extends this class, we are definiting a new instance of the WebSecurityConfigurer interface. We are overriding one of its methods configure(HttpSecurity http) to have our custom version of how we want to protect our endpoints.
configure(HttpSecurity http): This is the method we want to override to tell Spring Security that we only want to protect one of our two endpoints and not both, which would be the default behavior if we don’t do this.
HttpSecurity http: The input parameter of the method is the one containing all the configuration that Spring Security will consider during the execution of all the Filter Chain to any requests coming to the application.
formLogin(): Enables the Login (default) screen for any request to our protected endpoint
authorizeRequests(): The restrictions to the requests
antMatchers: Indicates the method and the endpoints (patterns) for each restrictions we want to give
authenticated() and permitAll(): I think they are self-explainatory. But basically, we specify which antMatcher we want to retrict to only be accessed if it’s authenticated user or not

Run the application

Let’s run the mvnw script once again and test the two endpoints in the browser to see how they behave with the given configuration.

$ ./mvnw spring-boot:run
# Note: The autogenerated password changes every time you run this command. This time I got:
Using generated security password: 5e57b493-6181-4682-bc21-81ec40222298

Try to open the public endpoint: http://localhost:8080/public-resource

The public resource doesn’t require any authentication process, which is expected since we are telling Spring Security to allow ALL requests to access it with this matcher: .antMatchers(HttpMethod.GET, "/public-resource").permitAll().

Now let’s try with the protected endpoint: http://localhost:8080/protected-resource

This time, right after we hit the URL in the browser, a login page is displayed. The login page was intentionally configured by giving the formLogin() to the http variable within the configure(...) method in our SecurityConfiguration class.

Now lets use the username by default and the password printed in the logs when we started the application.

By entering the correct credentials, the application will allow the user to access the protected resource. If we give incorrect credential, then the login page will let us know, like so:

Final thoughs

There are a lot of security concepts that I still need to study. Somewhat during this process of learning Spring Security I am exposed to understand some of those security definitions to apply the configurations accordingly.

I first tried to read all the official docs at once from Spring Security, but it was quite overwhelming, and I then preferred to create a project, and do things that later forced me to investigate more specific topics by the time I was facing some issues with the setup.

For now, I feel I learned few concepts that will allow me to expand later, perhaps in future articles.

Refs:

Github Repo: https://github.com/ckinan/learning/tree/main/java/spring-security-01
Spring Security Reference Docs: https://docs.spring.io/spring-security/site/docs/5.3.2.RELEASE/reference/html5/
Spring Security - Its 15 minutes tutorial: https://spring.io/guides/gs/securing-web/
I strongly recommend to read this article, which helped me a lot to understand some foundations of Spring Security: https://www.marcobehler.com/guides/spring-security
Also, I found the Spring One videos on Youtube very informative: https://www.youtube.com/watch?v=8rnOsF3RVQc&t and https://www.youtube.com/watch?v=WbnuwpSBXPs

Tailwind CSS - First Steps

cesar@cesarkina.com (Cesar Kina) — Sun, 17 May 2020 00:00:00 +0000

In my whole career, even when I was in college, every time I had to build an application that involves UI or styles, I always struggled with the design. After some years now, I still have the same problem. Colors, text size, layouts, etc are simply a nightmare to me. This is one of the things I usually try to avoid in web development when there is some CSS file I have to (inevitable) make some changes, I just say “oh God, please help me”.

Introduction

Ultimately, I’ve been hearing a lot about Tailwind CSS, a utility-first framework that comes with a bunch of amazing stuff for your design process, like for example predefined attributes to create a nice green button with rounded corners and a hover effect, but Tailwind doesn’t give you the whole class for the button, instead it brings with the necessary little pieces that you have to combine nicely to build the design for that button.

This is an example of how a green button using Tailwind CSS would look like:

<button
type="button"
class="rounded bg-green-500 hover:bg-green-700 text-white font-bold py-2 px-4 border-solid border-2 border-green-600 m-5">
Login
</button>

Someone might say, “why would I need that if there are other tools or frameworks that provide the whole button component styles out of the box”. Well, that was me. But, after giving it a try, I noticed that you can get exactly what the other frameworks can give you, but now you know exactly what you are including in your styles. Meaning, you get full control of how things are presented in your UI.

Motivation

There are a couple of things I really like about Tailwind CSS:

Documentation: Nice examples, well explained and I go there every time I need to do an isolated thing before doing it in my project.
Easy to get started: When I want to try something new, one of the first things I look for is the “Get Started” section. Well, here it is Get Started. I went to the “Using CDN” section because I just wanted to get a quick demo to explore.
Community: I see more and more people using it and sharing their projects or knowledge about it. One thing that convinced me to take a look at it was one guy replicating a whole application using Tailwind CSS. I was like “what? is that possible?”, so cool.

Another thing I got from there, and this might be something more “personal”, is that after I started using it I got myself more interested in to know more properties I can use in CSS, because, Tailwind can give you the utilities you need, but in order to use them, you have to know how you would accomplish that with plain CSS, then go to Tailwind docs and look for it without creating a single CSS line, you have everything you need available to use in your HTML.

It’s like my development process, in terms of design, changed quite a bit. I now feel I have more control of how I build my UI. Now, my steps to accomplish some styling are (example):

What do I need? I need a green button for my login page. Specs: Green background, white text, border and rounded corners.
How is this in plain CSS? Example for the green background: background-color: #48bb78; . I think this is the most difficult step at least in my case, there are a lot of things in CSS that are unknown to me.
Go to Tailwind CSS docs and search for the CSS property. Link: https://tailwindcss.com/docs/background-color . For the Green Background, the Tailwind CSS “equivalent class” should be class="bg-green-500"
Isolate the code and test. You can also test directly in your project, but I strongly recommend to test any style in a separate environment, so if something fails, you are not confused by the question “is it because of something in my project or is it with Tailwind or is it me doing something wrong?” (you know what I mean…) Example:

<!DOCTYPE html>
<html>
<head>
<link
rel="stylesheet"
type="text/css"
href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/1.4.6/tailwind.min.css"
/>
</head>
<body>
<button
type="button"
class="rounded bg-green-500 hover:bg-green-700 text-white font-bold py-2 px-4 border-solid border-2 border-green-600 m-5"
>
Login
</button>
</body>
</html>

Incorporate this in your project. You should make the changes according to your project needs once you get the desired results in your isolated tests.

Example in codepen: https://codepen.io/ckinan/pen/JjYePyW

Final thoughts

Tailwind CSS is a different way to see design in my development process. It didn’t magically solve all my problems with the design, and I don’t see it as something that I needed to do my styles looking better, but it kind of helped me to do more things directly in the HTML files, instead of doing it in CSS files just to make my design unique. Finally, the feeling of having control of every single detail in how my components look like without having to write any much CSS, keeps me motivated to continue utilizing Tailwind.

Project Github Pull Request Viewer

cesar@cesarkina.com (Cesar Kina) — Fri, 24 Apr 2020 00:00:00 +0000

How I built a pull request viewer using Github API. What I learned and what I failed.

Introduction

I am maintaining a software that its repo is in Github. That means, I had to get familiar with the tools this platform provides. Code Reviews with Pull Request, which is one of the features Github has, where you can interact with who owns the code changes before merging a branch into master.

As a reviewer, I have to open pull requests regularly to do my review. Meaning, more than likely I have to visit https://github.com/pulls frequently to see whichh PRs I have in my queue.

After some time seeing my list of PRs, I noticed there are cases where I would like to see more information in that list before clicking on one of those. Like for example, the number of approvals each PR already have, or if there is already activity such comments, requested changes, and who did which action, so with all these information in that same list I can have a clearer idea what’s going on there. Yeah, I know it’s only one click away from seeing all of that, but as I was about to study the Github API, I decided to give it a try.

Before going to the details, I want to start showing two images that somewhat explains what is this all about.

Official Github Pull Requests View looks like this:

My clone looks like this:

Expectations

The need

A different way to see the list of pull requests that could give me a better visibility of which code reviews I have in my queue and need my attention.

The approach

Make use of Github API to read my pull requests based on my username and show valuable information within a web application.

The (minimum) value

At least, I wanted to have:

Last updated date-time
Counts for current reviewer who: approved, requested changes, commented, awaiting within each pull request
Events in a timeline

There are some more “features” I included in this project for different reasons, like Authentication (for obvious reasons), a search form, and some basic filtering functionality.

The Jorney

The Stack

It was the time to choose the stack and tools to develop this web application. Here is the summary of what I got and why:

The Stack	The Why
ReactJS	I am not a frontend guy, but I had some little projects (mostly TODO apps) that I had built previously and this time I wanted to create something "more elaborated" and get a better understanding of the basics.
Github API	A must if I wanted to read data from Github
Netlify Functions (Javascript)	I needed a backend service to hide the Secret Key to make calls to Github API. I was not sure which service and programming language to choose. My first option was to create a Java or NodeJS service and deploy it on Heroku, but I didn't want to pay for this to be "live" forever (no sleep time). I wanted this to be totally free. That is why I decided to go with Netlify Functions, which doesn't have that kind of restrictions and it's free, at least for this small project that won't have too much calls, I said "it's good".
Primer CSS	I didn't want to spend time styling this web application. I literally just googled "github css styles" and found this cool thing that saved me a lot of time.

The process

1. Github API (Preliminary Research)

First of all, I had to do some research about how to consume Github API, there are some previous articles I posted about it:

By doing that, I was able to confirm if what I wanted to do was possible. Meaning: Reading Pull Requests from the API was viable.

2. Create the frontend

I was surprised how easy is to create a brand new web application and get it running. The create-react-app CLI was my guy and made me forget all my setup problems I had some time ago while following some tutorials that are well explained but I guess I am not that smart.

3. Create the backend

I had some reads about how use Netlify Functions to accomplish specific goals:

Goal #1: How to use Environment Variables. This is actually the most important one, because I wanted to have the Github Secret Key hidden in one of these Variables.
Goal #2: How to use its “develop mode”. Ref: https://www.netlify.com/products/dev/
Goal #3: How to handle cookies (my limited knowledge about secury led me to simply pass tokens using a cookie).

4. Connect the front with the back

It was the most interesting part. From here I learned few things:

A) GraphQL

This is the best thing I’ve discovered within this project. The first version of this project was to use the Github API v3, whis expose the endpoints with REST.

The only one thing that made me switch to v4 (GraphQL) was the number of requests I had to make to get Pull Requests information. Why is that? Because every time I wanted the list of pull requests and their details I had to make a Request to the endpoint to fetch the users, then the repos, then the pull requests, then the reviews, and so on. Nested loops everywhere, without mentioning that the JSON responses had data that I didn’t even need.

With GraphQL, I only had to be worried about one single endpoint and try my queries I previously tested within the GraphQL Explorer.

After switching to GraphQL I finally got the idea of using it, and I am pretty sure there are pros and cons about it, but at least for this project, it was a must.

B) I do know nothing about security

While developing the interface of my Netlify Functions, I was thinking how to prevent users from executing my APIs without any permissions. There were a lot of techniques and approaches I was not able to understand. I had to apply maybe one of the easiest ways to avoid being attacked by something from the client side. My method was basically the use of cookies to store the token after being authenticated.

I also developed two ways to be “authenticated” and consume the API:

Method #1: Authorizing OAuth Apps. Ref: https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/
Method #2: Store a Personal Access Token as Environment Variable. (Only meant to be used locally)

~~I believe this is the main reason I can’t get this project deployed in the cloud, because I can’t figure this thing out yet…~~ . UPDATE: Deployed on Netlify: https://gh-pr-viewer.netlify.app/

5. Test

After having the minimum things already developed, I forced myself to move from the official Github PRs view to my start using my application. It was a good move, I mean, things were going well, I had the information I originally said I need, few bugs but working.

During this stage, I had to refactor several parts of the code. Here I learned a little more about React Hooks, React Router, TypeScript and got used to this ecosystem more than I initially expected. I know I am not in the position to say I am now a “React” guy, but good enough to connect pieces and understand what’s going on with the components and data state flows. However, I can’t say “I know React”, because I am pretty sure the code of this app has a lot of “spaghetti” and it’s not using React in the right way.

Reality

So far so good. I have a web application, running in my local machine, reading pull requests information directly from the API. I actually use this daily to see what I have in my queue to do code review.

However, I don’t think I accomplished my original need: “Better visibility of my code reviews”. I have a web application that just show me things in a “different way”, but doesn’t make any difference in comparison with the official view from Github.

Final thoughts

In terms of “learnings”, I gained a lot from this project, because I was able to get some ReactJS skills, discovered one of the advantages of using GraphQL, and made some things with Serverless Functions (Netlify). But in terms of “solving my needs”, I think I failed, because it is irrelevant which view use (the Github one or mine), this app doesn’t make any difference at least in the current state.

Using something you are creating in your daily basis speeds up the development, because you realize what are the things you need to do immmediately or features you want to have or the bugs you need to fix, it’s like you are your own client, which keeps you motivated.

Moving forward, I’ll be still making more updates to this project, but for now, I don’t think this is going to be as regular as it was the past weeks.

Github API - OAuth tokens for apps

cesar@cesarkina.com (Cesar Kina) — Fri, 13 Mar 2020 00:00:00 +0000

How to use OAuth tokens for apps with Github API to authorize the access of certain data on behalf of the logged in user.

Github Web flow

Quoting Github docs:

Tokens should be created via a web flow. An application sends users to GitHub to log in. GitHub then presents a dialog indicating the name of the app, as well as the level of access the app has once it’s authorized by the user. After a user authorizes access, GitHub redirects the user back to the application.

Note: I am going to use what they call “web flow” because seems to be the recommended method and some other methods will be deprecated by the end of this year: https://developer.github.com/changes/2020-02-14-deprecating-oauth-auth-endpoint/

Example (Step by step)

In this example, I will write a simple web application that displays the list of all public repos and all their pull request of a user by following three steps.

Summary:

Authenticate Github API via web flow
Get repos of the authenticated user
Get pull requests per each repo

Details:

Authenticate Github API via web flow

Follow these steps to create an oauth app: https://developer.github.com/apps/building-oauth-apps/creating-an-oauth-app/ More information about oauth apps: https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/

In this example, this is the information I have:

Application name: OAuth App Local
Homepage URL: http://localhost:8888
Application description: Just left in blank
Authorization callback URL: http://localhost:8888

Then I created a simple index.html file with a single link that points to https://github.com/login/oauth/authorize?scope=user:email,read:org&client_id=<CLIENT_ID> . Notice <CLIENT_ID> in the URL is the client id that Github gives you when you create your OAuth App. This can be in your public code. However, later in this article we will use also the Client Secret, this SHOULD BE totally hidden from public access, meaning: it MUST BE in your backend layer.

Note: We are asking for the following “permissions”: user:email and read:org. Ref: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/

Once the user is authenticated through Github Login Page, Github will redirect to the “callback URL” including a code as query param like: http://localhost:8888?code=xxxxxx. This code will be used to make another call to get the access code.

The public site makes a call to the Netlify Function located in the same project under the path /.netlify/functions/github-client?code=<CODE>. Where <CODE> is what Github had sent to the callback url in the previous step.

The Netlify Function will make a call to https://github.com/login/oauth/access_token sending:

Client Secret (Environment Variable)
Client ID (Environment Variable)
Code (Query Param)

Note: For more information about Netlify Functions, read this: https://cesarkina.com/blog/netlify-function/

Once you get the token from the endpoint above, then you will be able to make calls to any Github API that your token has access, in this case, we asked for the following scopes: user:email and read:org and all public scopes that don’t need to be specified for authorization.

Get repos of the authenticated user through GET /user/repos endpoint . Ref: https://developer.github.com/v3/repos/#list-repositories-for-the-authenticated-user

Now that we have available the access token, we can make calls from the Front End to the Github API.

const showRepos = async () => {
fetch("https://api.github.com/user/repos", {
method: "GET",
headers: {
Authorization: "Bearer " + localStorage.getItem(GH_ACCESS_TOKEN_KEY),
},
})
.then(function (response) {
return response.json()
})
.then(function (data) {
document.getElementById("result").innerHTML += "<br/><br/>Repos"
data.forEach(function (repo, index) {
showReview(repo)
})
})
}

Get pull requests per each repo retrieved from previous step through GET /repos/:owner/:repo/pulls endpoint. Ref: https://developer.github.com/v3/pulls/#list-pull-requests

const showReview = async repo => {
fetch("https://api.github.com/repos/" + repo.full_name + "/pulls?state=all", {
method: "GET",
headers: {
Authorization: "Bearer " + localStorage.getItem(GH_ACCESS_TOKEN_KEY),
},
})
.then(function (response) {
return response.json()
})
.then(function (data) {
document.getElementById("result").innerHTML +=
'<br/><a href="' +
repo.html_url +
'" target="_blank">' +
repo.full_name +
"</a>"
data.forEach(function (pr, index) {
document.getElementById("result").innerHTML +=
'<br/><a href="' +
pr.html_url +
'" target="_blank">---- #' +
pr.number +
": " +
pr.title +
" (" +
pr.state +
")" +
"</a>"
})
})
}

Test

Redirected to Github Login Page:

You need to authorize this app:

Final result:

Final thoughts

I can see Github OAuth App web flow as standard way to authorize an app to read/write information on behalf of an authenticated user using an access tokens. Speaking particularly about Github, the only thing I am kind of dissapointed is the fact I can’t read Organization’s data without the permission of the owner, which make totally sense, but originally I thought it could be possible since I was able to access that kind of information via Personal Access Token, but seems like scope are pretty different and I should have read carefully the docs in the first place.

I would like to create a more elaborated app that can show more information about repos, pull requests and stats using what I’ve learned here.

Github repo: https://github.com/ckinan/learning/tree/main/ckn-github-oauth-app

Refs

Netlify Functions

cesar@cesarkina.com (Cesar Kina) — Sat, 07 Mar 2020 00:00:00 +0000

Exploring Netlify Functions: Concepts and some examples to get started with serverless functions.

Motivation

Sometimes I want to build simple web applications for demos or just for fun. These ideas relates interactions with APIs (free ones in most of the cases). However, even when these APIs have free plans, they force you to create an API KEY, which you don’t want to expose for security reasons.

I don’t want to use secret keys in a single font end web application, because that approach would expose my keys. So, I have to handle those in a server side (backend apps). And here is where serverless functions caught my attention.

There are some options I was checking to create and use serverless functions, but as I am already working with Netlify for this website, I wanted to give it a try and use “Netlify Functions”.

What is “Netlify Functions”?

Serverless functions on AWS Lambda, handled by Netlify. Create, build, deploy and test these serverless functions on Netlify and locally with some great tools that Netlify provides.

Currently, supported languages are:

JavaScript
Go

Concepts

To get started with Netlify Functions, I’ve found some particulars to create, configure and serve your functions.

netlify.toml (configuration file)
Structure of a function (obviously)
Commands to build and serve (the latter is optional if you want to deploy it locally)

Lets check some few details about them.

netlify.toml

This file contains the configurations of your functions project. For this (very) small example, I am just specifying where the built functions are located (“dist” folder).

Ref: https://docs.netlify.com/configure-builds/file-based-configuration/

Example:

[build]
functions = "dist" # build output will be here

Structure of a function

From a “hello world” example found here: https://functions-playground.netlify.com/

exports.handler = function (event, context, callback) {
callback(null, {
statusCode: 200,
body: "Hello, World",
})
}

Declare exports.handler function with the following parameters:

event : Requests data
context : Context of the call. Example: Functions with Netlify Identity
callback : Creates reponses

Commands to build and serve

netlify-lambda build : Builds your function source code and place it into your built folder specified in the netlify.toml configuration file in functions. Ref
netlify-lambda serve : Runs your function locally. Ref

Getting started (example step by step)

Initialize your project

npm init

Install netlify-lambda command locally

yarn add netlify-lambda

Create the netlify.toml file

[build]
functions = "dist" # build output will be here

Add build and serve scripts to package.json. Note our source code folder is in ./src path.

 :
"scripts": {
"build": "netlify-lambda build src",
"install": "netlify-lambda install src",
"serve": "netlify-lambda serve src"
},
:

Create your first function and name the file as hello-world.js.

exports.handler = function (event, context, callback) {
callback(null, {
statusCode: 200,
body: "Hello, World",
})
}

Your source tree should look like:

Deploy

Two options: Locally and on Netlify

7.1. Option 1: Locally

npm run serve

You will see your port where your local function is exposed:

[0] ./hello-world.js 129 bytes {0} [built]
Lambda server is listening on 9000

Open in a browser: http://localhost:9000/.netlify/functions/hello-world

Note: The path of the URL is: http://{host}:{port}/.netlify/functions/{your-function-name}) . UPDATE: I tried to change the .netlify/functions path thing, but it was not possible. But, it is possible to redirect requests asking for your functions to whatever alias you want to have like /api for instance. See: Netlify Routing Redirects.

You will see the response of the function: “Hello, World”

7.2. Deploy on Netlify

Commit and push your project into Github/Gitlab/Bitbucket, and deploy your project in Netlify, where the “Build Command” should be in this case: npm run install. Once it got deployed, you can see the list of functions in Netlify under Project > Functions.

Click on any of your functions, you will see the URL to invoke it. Example

Note: There are really good examples provided by Netlify Team of what you can do within a single function. I actually used them for this article. Link: https://functions-playground.netlify.com/

Logging

Quoting docs: https://docs.netlify.com/functions/logs/

Netlify displays a running log for the last hour of activity, including:

Start of each invocation

Any console.log() statements you include in your function code

Environment variables

Following another example from https://functions-playground.netlify.com/ to get environment variables, create env.js under src folder.

const GREETING = process.env.GREETING
exports.handler = async (event, context) => {
return {
statusCode: 200,
body: GREETING,
}
}

Here, GREETING is your environment variable. This is how you “hide” certain values you don’t want to share publicly to your clients behind a serverless function like this. Even if your source code is public in Github, you won’t need to have secret information in your code, but in environment variables like so.

In the above example, if you are running:

Locally: Follow instructions in this link https://github.com/netlify/netlify-lambda/issues/118#issuecomment-506973346
On Netlify: Just create an environment variable under Project > Settings > Build & Deploy > Environment option.

Final thoughts

Netlify Functions is definitely a great choice to get started with serverless functions and it will (for sure) help me to accomplish what I was looking for: to hide api secret keys to invoke third party’s APIs (more than likely I’ll reuse the “fetch.js” example).

I am pretty sure there a lot more features Netlify provides within Functions behind the scenes happening that I am not aware of right now. But, I am happy with what I’ve just explored and learned about it, most importantly, Netlify is clear in their documentations and has a great community out there with super easy tutorials for beginners (like myself).

Github repo: https://github.com/ckinan/learning/tree/main/ckn-netlify-functions

EDIT: Here is another example using Netlify Functions and this free API to get the weather: https://weatherstack.com/ . Repo: https://github.com/ckinan/learning/tree/main/ckn-netlify-weather

References

Official docs: https://docs.netlify.com/functions/overview/
netlify-lambda: https://github.com/netlify/netlify-lambda
Local environment variables https://github.com/netlify/netlify-lambda/issues/118#issuecomment-506973346
Great article to get started: https://kentcdodds.com/blog/super-simple-start-to-serverless
Another great article: https://flaviocopes.com/netlify-functions/

Github API - First Steps

cesar@cesarkina.com (Cesar Kina) — Fri, 07 Feb 2020 00:00:00 +0000

First steps with Github API, looking at the official docs, executing some APIs and discovering what I can do with that.

Motivation

Currently, the company I am working with uses Github as git repository/platform. Code reviews, documentation, merges and other stuff going through there. It’s going great.

I am curious about what else we can do using Github API, what pieces of the whole development process we can automate. So, this article is about Github API, what I know, what I learned, what I find useful.

I may use the outcome of this (quick) research not only for this company, but anything else that may help other problems/opportunities that, right now, I can’t even imagine.

Steps

This is the list of things this article/research will cover:

Step #1: Take a look at their docs (overview)
Step #2: Understand their authentication processes
Step #3: Endpoints

Basics

Docs! Lets see how they explain their API in their documentation and see how easy (or complex) is to understand what we can achieve with it.

Link: https://developer.github.com/v3/guides/getting-started/

This guide has very clear examples of how to call the API, like this one that doesn’t require any auth information and reads all public data from any github user using cURL:

$ curl https://api.github.com/users/ckinan
{
"login": "ckinan",
"id": 7999613,
"node_id": "MDQ6VXNlcjc5OTk2MTM=",
"avatar_url": "https://avatars1.githubusercontent.com/u/7999613?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/ckinan",
"html_url": "https://github.com/ckinan",
"followers_url": "https://api.github.com/users/ckinan/followers",
"following_url": "https://api.github.com/users/ckinan/following{/other_user}",
"gists_url": "https://api.github.com/users/ckinan/gists{/gist_id}",
"starred_url": "https://api.github.com/users/ckinan/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/ckinan/subscriptions",
"organizations_url": "https://api.github.com/users/ckinan/orgs",
"repos_url": "https://api.github.com/users/ckinan/repos",
"events_url": "https://api.github.com/users/ckinan/events{/privacy}",
"received_events_url": "https://api.github.com/users/ckinan/received_events",
"type": "User",
"site_admin": false,
"name": "Cesar Kina",
"company": null,
"blog": "https://ckinan.com/",
"location": "Lima, Peru",
"email": null,
"hireable": null,
"bio": "software developer",
"public_repos": 22,
"public_gists": 1,
"followers": 3,
"following": 7,
"created_at": "2014-06-26T20:18:31Z",
"updated_at": "2020-02-08T00:08:38Z"
}

However, like any other API, it has a limited number of requests that any client can make to the API without being authenticated. Quoting:

Unauthenticated clients can make 60 requests per hour. To get more requests per hour, we’ll need to authenticate. In fact, doing anything interesting with the GitHub API requires authentication.

Authentication types

Basically, 2 options:

Option #1: Using personal access tokens: https://developer.github.com/v3/guides/getting-started/#using-personal-access-tokens

The easiest and best way to authenticate with the GitHub API is by using Basic Authentication via OAuth tokens. OAuth tokens include personal access tokens.

Examples:

curl -i -u <your_username>:<your_access_token> https://api.github.com/user/repos

Provide privileges to your token to read your repos.

Option #2: Using OAuth tokens for apps : https://developer.github.com/v3/guides/getting-started/#using-oauth-tokens-for-apps

Apps that need to read or write private information using the API on behalf of another user should use OAuth.

Endpoints

All the endpoints of the API in their documentation have examples of the calls, details of the input parameters and show how the return payload should look like, which is pretty good.

Ref (overview of the API): https://developer.github.com/v3/

Docs give the endpoints paths like this (not the full url).

Example of user repos: GET /user/repos

These are just 3 of the endpoints I tried out using a personal access token I’ve created:

Get repos
Get pull request
List reviews of a pull request

Here more details about each of them:

Get repos

Endpoint: GET /user/repos

Ref: https://developer.github.com/v3/repos/#list-your-repositories

Example:

$ curl -i -u <your_username>:<your_access_token> https://api.github.com/user/repos

Get pull requests

Endpoint: GET /repos/:owner/:repo/pulls

Ref: https://developer.github.com/v3/pulls/#list-pull-requests

$ curl -i -u <your_username>:<your_access_token> https://api.github.com/repos/ckinan/ckinan.com/pulls?state=all

Parameters are also available, like state in the example above to list “Either open, closed, or all to filter by state. Default: open”

List reviews of a pull request

Endpoint: GET /repos/:owner/:repo/pulls/:pull_number/reviews

Ref: https://developer.github.com/v3/pulls/reviews/#list-reviews-on-a-pull-request

$ curl -i -u <your_username>:<your_access_token> https://api.github.com/repos/ckinan/ckinan.com/pulls/9/reviews

This is interesting, because you can see who submitted the review, whether this review refers to a comment only, request changes or approval.

Final thoughts

Github has a great documentation, at least for a person who is obviously just trying to get used to the API (like me).

For sure, using personal access token is easier to get used to the API. You just need to create one by going to your Settings and include your token in your calls. But, depending on the use case, it may be annoying to ask all users to create another token to make the application working, in that case, OAuth tokens for apps seems to be a better option, so that application can get authorization on behalf of the user who is logging in without explicitly creating a personal access token.

I will have another space to continue this investigation, where I’d like to see more about OAuth tokens for apps, and play with the endpoint to read pull requests.

Auth0 - Simple Example with Vanilla JS

cesar@cesarkina.com (Cesar Kina) — Sun, 05 Jan 2020 00:00:00 +0000

First steps with Auth0 following their tutorial to create a simple login page with Vanilla JS.

Intro

Auth0 is a complete platfom that helps you to handle authentication and authorization services in your application. It provides SDKs to connect to their solution for many technologies such as Angular, Javascript, React Vue for SPA applications, not to mention Node, PHP, Python, Java (and others) for backend and others for mobile apps.

This post is intended to get started with Auth0 by following the steps of the tutorial they have in their website using Vanilla JS (which is great and very detailed). However, after I completed all the steps, I decided to remove some parts of the code from the code, because all I wanted to have there was the most minimal version of this login page just to avoid being distracted by other things that are not directly related to the Auth0 SDK. During my research, I found this post which inspired me to do this work: https://protoout.studio/posts/auth0-github-pages/ . So, thanks Protoout for this :)

Quoting the goal of this tutorial:

This tutorial demonstrates how to add user login to a Javascript application using Auth0.

You can find the tutorial here: https://auth0.com/docs/quickstart/spa/vanillajs

After executing all the steps in there I got this configuration in my first application I called auth0-javascript-tutorial.

Steps

Auth0 Settings

Domain and Client ID, you will need them later in your JS app.

Configure the following:

Allowed Callback URLs
Allowed Web Origins
Allowed Logout URLs

One Auth0-Application supports many Allowed Callback URLs, Web Origins and Logout URLs, so I have set only one application to accomplish multiple environments (local and github.io in my case).

index.html

Simple html file to show login/logout buttons and the gated-content section where the logged user profile information should be displayed. No need to change anything here, keep it as the tutorial suggests it to be.

You can load the auth0-spa-js from CDN.

<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<title>SPA SDK Sample</title>
<style>
.hidden {
display: none;
}
label {
margin-bottom: 10px;
display: block;
}
</style>
</head>
<body>
<h2>SPA Authentication Sample</h2>
<p>Welcome to our page!</p>
<button id="btn-login" disabled="true" onclick="login()">Log in</button>
<button id="btn-logout" disabled="true" onclick="logout()">Log out</button>
<div class="hidden" id="gated-content">
<p>
You're seeing this content because you're currently
<strong>logged in</strong>.
</p>
<label>
Access token:
<pre id="ipt-access-token"></pre>
</label>
<label>
User profile:
<pre id="ipt-user-profile"></pre>
</label>
</div>
<script src="https://cdn.auth0.com/js/auth0-spa-js/1.2/auth0-spa-js.production.js"></script>
<script src="app.js"></script>
</body>
</html>

app.js

This is how my minimal version of the app.js looks like:

I am not serving the Domain and Client ID from any backend service (as the tutorial does serving them using Express).
Just including them in the client as I don’t care about them being exposed publicly for now.
Then I organized the onload() function just in three steps: Configure the Client, Process the Login State and Update the UI.
I made sure I didn’t miss any Auth0 function from the SDK that was originally used in the tutorial, so that I can check them out later.

let auth0 = null
window.onload = async () => {
await configureClient()
await processLoginState()
updateUI()
}
const configureClient = async () => {
auth0 = await createAuth0Client({
domain: "dev-scp6vo-0.auth0.com",
client_id: "7uwc6i4z1Xxn9JpWaz2NU1bc9uq4oBPw",
})
}
const processLoginState = async () => {
// Check code and state parameters
const query = window.location.search
if (query.includes("code=") && query.includes("state=")) {
// Process the login state
await auth0.handleRedirectCallback()
// Use replaceState to redirect the user away and remove the querystring parameters
window.history.replaceState({}, document.title, window.location.pathname)
}
}
const updateUI = async () => {
const isAuthenticated = await auth0.isAuthenticated()
document.getElementById("btn-logout").disabled = !isAuthenticated
document.getElementById("btn-login").disabled = isAuthenticated
// NEW - add logic to show/hide gated content after authentication
if (isAuthenticated) {
document.getElementById("gated-content").classList.remove("hidden")
document.getElementById(
"ipt-access-token"
).innerHTML = await auth0.getTokenSilently()
document.getElementById("ipt-user-profile").innerHTML = JSON.stringify(
await auth0.getUser()
)
} else {
document.getElementById("gated-content").classList.add("hidden")
}
}
const login = async () => {
await auth0.loginWithRedirect({
redirect_uri: window.location.href,
})
}
const logout = () => {
auth0.logout({
returnTo: window.location.href,
})
}

The above code snippet is a minified version of what Auth0 has prepared in their documentation. Basically, it establishes connection with the Application I created in their platform, then do some checks with the state of the authentication process to show or hide information about the user who is (or not) logged in. Below is the list of classes & methods that stricly come from the Auth0 SDK:

createAuth0Client
handleRedirectCallback
isAuthenticated
getTokenSilently
getUser
loginWithRedirect
logout

This exercise helped me to have a better understanding what actually this simple login page is doing through the SDK. Also, the SDK documentation has good content about each class/function/property involved here: https://auth0.github.io/auth0-spa-js/

One key point in app.js is this line:

await auth0.handleRedirectCallback()

According to the SDK documentation:

After the browser redirects back to the callback page, call handleRedirectCallback to handle success and error responses from Auth0. If the response is successful, results will be valid according to their expiration times.

Whatever happens with the authentication process won’t be effectively readable in your application until the call to auth0.handleRedirectCallback();, you won’t be even able to get the actual state of the login process result. For example: auth0.isAuthenticated(); would not get true after a successful login if it was not handled previously by the auth0.handleRedirectCallback(); function.

Test

Open https://ckinan.github.io/auth0-javascript-tutorial/ and click on the “Login in” button.

You will be redirected to Auth0 Authentication page where you can login to this app using your Google or Github account. By the way, the free plan allows you to set up to two social connections.

After login success, it will redirect to your page and the user information will be displayed.

Conclusions

Auth0 is a great option to handle Authentication and Authorization processes. Their tutorial is very detailed and easy to follow (didn’t have much problems). However, in order to focus on what really matters to me in their SDK, I had to remove/debug/reinsert some lines of code. I am excited about building a more real use case using Auth0 and see more features of it.

Intro

cesar@cesarkina.com (Cesar Kina) — Tue, 08 Oct 2019 00:00:00 +0000

This is the first article of this website. As software engineer, I would be writing stuff related to that field (mmhh or maybe not). For now, I just want to mention the awesome tools I found to build this website.

Static site generator

Gatsby
UPDATE December 2019: Moved to Docusaurus
UPDATE April 2020: Came back to Gatsby
UPDATE January 2021: Moved to my own static site generator

Hosting

Netlify
UPDATE January 2021: Moved to Digital Ocean
UPDATE early 2021: Moved to Netlify
UPDATE June 2021: Moved to Firebase
UPDATE January 2022: Setup Github Actions for site deployment

Styles

Plain CSS
UPDATE April 2020: Moved to Tailwind CSS
UPDATE January 2021: Plain CSS again
UPDATE July 2021: Dark theme, color palette from Nord Theme
UPDATE January 2022: Go back to light theme + High contrast

Final Notes

I am totally new to these technologies, so I might not be using them properly or not aware of all the features they provide. I simply wanted to start writing, so for now I’ll be using platforms and technologies built by more experienced people, then I’ll try to build this site from scratch by my own.

UPDATE Jan 2021: I decided to use this post to mention changes I’m doing to this site.