Comment by Louis Maddox on Google Big Query won't let me schedule query
The question is closed so I have to comment here: this can happen if your browser blocks popups, as the auth for creating/modifying scheduled queries uses a popup. If it's blocked, you see the...
View ArticleComment by Louis Maddox on Unable to create a scheduled query in BigQuery,...
No, you don't need to do this.
View ArticleComment by Louis Maddox on Is there a way in Python to do paragraph...
LexPredict has paragraph segmentation lexpredict-lexnlp.readthedocs.io/en/docs-0.1.6/modules/…
View ArticleComment by Louis Maddox on How to properly write cross-references to external...
There's a guide to the syntax of the inventory here: in particular the format of each line of object data goes {name} {domain}:{role} {priority} {uri}...
View ArticleComment by Louis Maddox on How to increase the maximum size of the AWS lambda...
You can get it down by building these (NumPy and SciPy) from source, I don't know if there's an easier way but see my comments in this thread for key parts of the Dockerfile I used: discuss.python.org/t/…
View ArticleComment by Louis Maddox on No module named 'pydantic_core._pydantic_core' in...
No you don't need to explicitly include pydantic or pydantic_core if using it via FastAPI. pip install fastapi lets import pydantic and import pydantic_core._pydantic_core run just fine. In fact the...
View ArticleComment by Louis Maddox on How do I add selenium & chromedriver to an AWS...
Not the case that selenium requires Dockerising your Lambda! I'll add an alternative approach as a separate answer, I got a zipped size of 70MB. Previously was 44MB (in the midst of an upgrade now, not...
View ArticleComment by Louis Maddox on FastAPI: How to specify possible values for a...
ConfigDict has a flag use_enum_values = True for this case docs.pydantic.dev/latest/api/config/…
View ArticleComment by Louis Maddox on Error with simple subclassing of pathlib.Path: no...
You don't need to do this (define the _flavour attribute) since Python 3.12 docs.python.org/3/whatsnew/3.12.html
View ArticleComment by Louis Maddox on Using Rust nightly in production
Note: Pyo3 moved to stable in 2020 github.com/PyO3/pyo3/issues/5
View ArticleHow to make vim code folding sync across splits?
I'm trying out code folding in vim but noticed it doesn't seem to mirror across pane splits when using scrollbind to achieve a "2-up" view of a long text/code file with relatively narrow lines (set up...
View ArticleAnswer by Louis Maddox for cross-compilation terminologies --- build, host...
"There are three system names that the build knows about: the machine you are building on (build), the machine that you are building for (host), and the machine that GCC will produce code for (target)....
View ArticleAnswer by Louis Maddox for Reading only a part of a large wav file with Python
There are a few packages you may want to look into for handling audio: commonly soundfile is used for I/O, as is librosa. The 'sampling rate' AKA 'frame rate' is the number of audio samples per second,...
View ArticleAnswer by Louis Maddox for R Markdown - variable output name
You can keep the simplicity of using the RStudio Knit button and reproducibility of a YAML header by using the undocumented knit hook to redefine what the button does (default function called is...
View ArticleAnswer by Louis Maddox for Is there an easy way to convert ISO 8601 duration...
Great question, obviously the "right" solution depends on your expectations for the input (a more reliable data source doesn't need as much input validation).My approach to parse an ISO8601 duration...
View ArticleAnswer by Louis Maddox for Dynamic inheritance in Python
As an alternative to Chris's answer implementing the memoisation suggestion for shx2's answer, I'd prefer to use a memoize decorator (the end result is still a class but it's clearer to me that the...
View ArticleAnswer by Louis Maddox for Python doctest how to match single or double quote...
I imagine you could achieve this with blacken-docs but I haven't seen an example of this done yet...Either the approach of:Leave the strings in black standard (double quoted) and find a way to...
View ArticleAnswer by Louis Maddox for How do I set up custom styles for...
I don't know which is most "official" but if you go to the "customisation" page of the Furo theme (developed by one of the Sphinx developers) and then scroll to "Custom CSS Files" it links to a guide...
View ArticleAnswer by Louis Maddox for Download only a range of bytes or a specific file...
Yes, it’s possible for an uncompressed tarball, the file format has header records about the files you can use to check its contents.I'm more of a Python than a Java guy, but take a look at my...
View ArticleAnswer by Louis Maddox for How to read a file in reverse order?
Here's a Python 3.8+ approach, using two string buffers, with grep-like substring matching (or just simply iterating each and every line if the empty substring is passed). I'd expect this to be more...
View ArticleAnswer by Louis Maddox for Asynchronous Requests with Python requests
I second the suggestion above to use HTTPX, but I often use it in a different way so am adding my answer.I personally use asyncio.run (introduced in Python 3.7) rather than asyncio.gather and also...
View ArticleAnswer by Louis Maddox for Close asyncio loop on KeyboardInterrupt - Run stop...
You want to add a signal handler as shown in this example in the docs:import asyncioimport functoolsimport osimport signaldef ask_exit(signame, loop): print("got signal %s: exit" % signame)...
View ArticleAnswer by Louis Maddox for python asyncio & httpx
Here's a nice pattern I use (I tend to change it a little each time). In general, I make a module async_utils.py and just import the top-level fetching function (e.g. here fetch_things), and then my...
View ArticleAnswer by Louis Maddox for Read webhook payload in Gitlab CI
If you run compgen -v to show the environment variables when triggering the pipeline in the UI (without JSON payload) you get 3 fewer lines in your job log than when POSTing a JSON payload.The...
View ArticleMultiple assignments via walrus := operator?
I've attempted to make multiple assignments with the walrus operator, and seen questions on StackOverflow such as this which also fail to assign multiple variables using a walrus operator, and am just...
View ArticleAnswer by Louis Maddox for Changing file permission in Python
In Python 3.4+, when working with pathlib.Path objects, you can call chmod() directly as a method on the path object. The following example is given in the docs:>>> p =...
View ArticleAnswer by Louis Maddox for GITLAB CI Error loading key "/dev/fd/63": invalid...
As mentioned in this thread on GitLab's bug tracker, the issue can arise when carriage return characters (\r) are added to the variable (a.k.a. "secret"). This can be worked around by piping to tr -d...
View ArticleAnswer by Louis Maddox for Git clone changes file modification time
To do this in Python is simpler than some of these other options, as os.utime accepts the Unix timestamp output by the git log command. This example uses GitPython but it'd also work with...
View ArticleAnswer by Louis Maddox for FFMPEG- Convert video to images
In addition to the select filter in Gyan's answer (which I use with eq rather than between), I came across another filter in the manual: thumbnailSelect the most representative frame in a given...
View ArticleHow to see logs of Dockerfile build when using custom AWS Lambda container image
AWS Lambda supports container images rather than using the standard Amazon Linux (2) image, but I can't figure out where the logs from the Dockerfile build are stored.I can find and see the logs for...
View ArticlePiping Rscript gives error after output
I wrote a small R script to read JSON, which works fine but upon piping withRscript myscript.R | headthe (full, expected) output comes back with an errorError: ignoring SIGPIPE signalExecution...
View ArticleCreate a new file in git bash
I've got Git for Windows running, I'm not sure if it's supposed to function as a text editor though?I think I installed it with the Vim editor, but in a Git Bash shell how do I create a file, such as...
View ArticleAnswer by Louis Maddox for Counting commas in a line in bash
An example Python command you could run (since it's going to be installed on most modern shells) is:python -c "import pathlib; print({l.count(',') for l in...
View ArticleAnswer by Louis Maddox for Convert text to bytes from Bash shell?
I adapted Machinexa's nice answer a little for my needsencoding="utf-8" is the default so no need to passmore concise to just import sys and use directlyHere I'm looking to make a unique set not a list...
View ArticleAnswer by Louis Maddox for how to grep ASCII control character
As suggested here, awk portably finds control charactersawk '/\01/'
View ArticleAnswer by Louis Maddox for python3 dataclass with **kwargs(asterisk)
All of these changes are well-meaning but pretty clearly against the spirit of dataclasses, which is to avoid writing a bunch of boilerplate to set up a class.Python 3.10 introduces the match statement...
View ArticleAnswer by Louis Maddox for Parse mailto urls in Python
I like Alexander's answer but it is in Python 2! We now get urlparse() and parse_qs() from urllib.parse. Also note that sorting the header in reverse puts it in the order: to, from, body.from...
View ArticleAnswer by Louis Maddox for Create a field which is immutable after being set
You now do this (in Pydantic v2) with the Field(frozen=True) kwarg (docs)Immutability ¶The parameter frozen is used to emulate the [frozen dataclass]behaviour. It is used to prevent the field from...
View ArticleComment by Louis Maddox on Pandas read_sql() - AttributeError: 'Engine'...
Known bug in 2.2 github.com/pandas-dev/pandas/issues/57049
View ArticleComment by Louis Maddox on how to install devtoolset-8 / GCC 8 on Amazon Linux 2
The URL is now 404, archived, I changed them to the current ones but the repo no longer works when used as shown
View ArticleComment by Louis Maddox on How can I update Google Colab's Python version?
Not working for Python 3.12, crashes and won't connect on refresh, I had to hard refresh. Appears to have broken something in Colab for this specific notebook! The message says something about not...
View ArticleComment by Louis Maddox on How to maintain Polars DataFrame metadata through...
This does not persist, with_columns (github.com/pola-rs/polars/blob/…) calls a classmethod (github.com/pola-rs/polars/blob/…) which calls __new__ not __init__
View ArticleAnswer by Louis Maddox for Polars: How to reorder columns in a specific order?
I wrote a plugin for this (on GitHub here), please try it out:pip install polars-permutePolars Permute PluginA Polars plugin for easily reordering DataFrame columns.Supports column permutations like...
View ArticleAnswer by Louis Maddox for Does Polars support creating a dataframe from a...
Interesting notes above, I had a look at how you do this and what the considerations are - I wrote a couple of notebooks on it:Notes on tree nestingReading highly nested tree data as Polars...
View Article