In last article, we tested the performance of single disk. However, in real world, disk can sometime fail, causing disk loss. RAID is created to protect data from disk failure using inexpensive disks. Generally there are hardware RAID and software RAID. This article is talking about software RAID. …

These days major cloud providers provide lots of managed services, even serverless computing, that remove you from the concern of managing infrastructure and caring about underlying hardware (CPU, memory, disk, network). However, there are still lots of legacy workload that assumes there is a server. …

Increase readability and robustness of your deep learning code — In previous article 1, 2, we reviewed Image classification with transfer learning with Tensorflow/Keras and PyTorch. Keras API is straightforward since Keras is high-level API (compared to low-level Tensorflow), however, PyTorch has lots of boilerplate code in the training function. Does PyTorch have something similar to Keras? …

These days when you talk about open source stream processing technologies, it is mostly Apache Spark, Apache Flink or Apache Beam. However, during old days, there is a technology called Apache Storm and lots of enterprises are still using that today and you cannot convert them all at once. This…

In previous article, we reviewed Image classification with transfer learning with Tensorflow/Keras. PyTorch is another popular deep learning framework, and the same pattern is applicable here. Again, the general steps in image classification transfer learning are: Data loader Preprocessing Load pretrained model, freeze model layers according to your needs Add…

How to troubleshoot problems in proper way — Operators are pieces of software that ease the operational complexity of running another piece of software. In Kubernetes/OpenShift world, operators can help operate the applications (e.g. installation, upgrading, advanced operators can also do backup, restore, HA/DR). Generally we should use provider-provided operator/custom resource definition, since it will be better supported…

Terraform is important IaC tool in multi-cloud context. I shared Terraform resources to get started on it. However, for any code, if you cannot test it, it is a big issue in DevOps world. Terratest is a popular tool to test terraform code. …

It is a hot trend to move to cloud, whether it is public cloud, private cloud, Kubernetes/OpenShift-based containers. A new paradigm 1, 2 of cloud-native application is also emerging. To leverage cloud computing model, you need to know what is available for you to leverage. …

In my previous Transfer learning post, we reviewed transfer learning for NLP. Huggingface has made NLP transfer learning very easy. However, so far, I have not found similar framework for various computer vision tasks. Let’s review image classification task to see what is the pattern. …