User:Kim/Stations, Skills, Resources/Git API

scheming around with the Git API
Resources:

can I use this to transform my personal reader into a (print / web to print) publication?

Issues

to mess around with my xp reader repository I created a new branch but this messed up all the curl requests so I decided to make a copy (not exactly a fork) using this command git push https://gitlab.com/km_kt/xp-reader-fork +gitpub:main (where xp reader fork is my newly initialized repo where i copy the branch gitpub from my old repo into and make it main) [1]
this actually worked! I could then simply clone it to local with http

Links

js

scrape content from dynamic websites uses python (and a lot of libraries >> is there a way around that?)

wget and curl dont run js (on a webpage) by themselves, this needs to be emulated by a pseudo browser (which is what they do above) using Selenium another option seems to be phantom.js as described here

cli / other

cat jp https://www.baeldung.com/linux/jq-command-json

terminal

these worked:
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/user
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects/69537327/repository/files/queue.md/blame?ref=main (this returns file content and commit information [2])

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" -o output.json (this returns commits for a specific file in json format)

jq

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" | jq '.[] | .message' this returns contents of message as strings

| jq -s '[ .[] | map(.) | .[] | {message: .message, created: .created_at}]' >> out.json (-s (for slurp) allows JQ to process the file as a single array object)

jq -r outputs raw text without " " like here | jq -r '.[] | .message,.created_at' >> out.txt

to print

similar steps have been done here Terms of servers

pandoc

this should get all files and subdirectories recursively
**/*.md -o gitpub.html

weasyprint

web (js)

this works for a simple fetch request in js (important are the origin: '*' to prevent CORS)

const url = 'https://gitlab.com/api/v4/projects/69537327/repository/commits';
fetch(url, {
    method: 'GET',
    origin: '*',
    headers: {
        'PRIVATE-TOKEN': 'glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln',
        'Content-Type': 'application/json',
     },
})

This Example fetches Commits for the file queue.md: https://hub.xpub.nl/cerealbox/~kim/test-3.html
Problem: this is a dynamic site, when trying to convert its contents (for printing to .md), nothing is there
-> tried Curl solution (see above), managed to curl same contents into json - now pandoc error when converting json into .md
pandoc -f csljson -t gfm queue-commits.json -o queue-commits.md this is the command i tried out