How to TCPdump in Kubernetes via ephemeral container

 kubectl debug -it ingress-nginx-controller --image=dockersec/tcpdump --target controller -- sh

 

https://cloudyuga.guru/blogs/how-to-tcpdump-in-kubernetes/

Build Airflow docker image with R

dockerfile:

FROM apache/airflow:2.5.1-python3.10
USER root
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B7B3B788A8D3785C
RUN apt-get update
RUN apt-get install -y r-base r-recommended default-jre default-jdk
RUN R -e "install.packages('RJDBC', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('gtools', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('data.table', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('dplyr', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('ABCanalysis', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('forecast', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('reshape2', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('mefa', repos = 'http://cran.us.r-project.org')"
RUN R -e "install.packages('rJava', repos = 'http://cran.us.r-project.org')"
USER airflow

 docker build -t apache/airflow:2.5.1-python3.10-R .

Algorithms and Data Structures Matter

Number of IoT devices: 90k

Volume: 24h of measures

 

DB engines:

PostgreSQL:  1346,89 GB

InfluxDB: 92,16 GB

VictoriaMetrics: 19 GB

Clickhouse: 2.75 GB

Query of last measures for all devices:

VictoriaMetrics: 20-40s

Clickhouse: 1.5-2s

Hardware for Deep Learning

Hardware for Deep Learning.ToC

 Hardware for Deep Learning. Part 4: ASIC

Reference design for 8-20 DGX A100 POD Ethernet storage network

Compute network:

    QM8790 40xHDR200

Storage network:

    SN3700 32x100GbE/ SN4600 64x100GbE

InBand network:

    SN3700 32x100GbE / SN4600 64x100GbE / SN2010 18x25GbE+4x100GbE

OOB network:

    SN2201 48xGbE+4x100GbE

The Square Kilometre Array Science Data Processor Preliminary Compute Platform Design

The Square Kilometre Array Science Data Processor Preliminary Compute Platform Design 

Hybrid Storage approache in HPC