1_DevOps'ish

AWS is open sourcing its Cedar policy language and authorization engine and Snapchange, an open source snapshot-based fuzzing tool.

Author : Sascha Grunert The Security Profiles Operator (SPO) is a feature-rich operator for Kubernetes to make managing seccomp, SELinux and AppArmor profiles easier than ever. Recording those profiles from scratch is one of the key features of this operator, which usually involves the integration into large CI/CD systems. Being able to test the recording capabilities of the operator in edge cases is one of the recent development efforts of the SPO and makes it excitingly easy to play around with seccomp profiles. Recording seccomp profiles with spoc record The v0.8.0 release of the Security Profiles Operator shipped a new command line interface called spoc , a little helper tool for recording and replaying seccomp profiles among various other things that are out of scope of this blog post. Recording a seccomp profile requires a binary to be executed, which can be a simple golang application which just calls uname(2) : package main import ( "syscall" ) func main () { utsname := syscall.Utsname{} if err := syscall.Uname (& utsname); err != nil { panic (err) } } Building a binary from that code can be done by: go build -o main main.go ldd ./main not a dynamic executable Now it's possible to download the latest binary of spoc from GitHub and run the application on Linux with it: sudo ./spoc record ./main 10:08:25.591945 Loading bpf module 10:08:25.591958 Using system btf file libbpf: loading object 'recorder.bpf.o' from buffer … libbpf: prog 'sys_enter': relo #3: patched insn #22 (ALU/ALU64) imm 16 - 16 10:08:25.610767 Getting bpf program sys_enter 10:08:25.610778 Attaching bpf tracepoint 10:08:25.611574 Getting syscalls map 10:08:25.611582 Getting pid_mntns map 10:08:25.613097 Module successfully loaded 10:08:25.613311 Processing events 10:08:25.613693 Running command with PID: 336007 10:08:25.613835 Received event: pid: 336007, mntns: 4026531841 10:08:25.613951 No container ID found for PID (pid=336007, mntns=4026531841, err=unable to find container ID in cgroup path) 10:08:25.614856 Processing recorded data 10:08:25.614975 Found process mntns 4026531841 in bpf map 10:08:25.615110 Got syscalls: read, close, mmap, rt_sigaction, rt_sigprocmask, madvise, nanosleep, clone, uname, sigaltstack, arch_prctl, gettid, futex, sched_getaffinity, exit_group, openat 10:08:25.615195 Adding base syscalls: access, brk, capget, capset, chdir, chmod, chown, close_range, dup2, dup3, epoll_create1, epoll_ctl, epoll_pwait, execve, faccessat2, fchdir, fchmodat, fchown, fchownat, fcntl, fstat, fstatfs, getdents64, getegid, geteuid, getgid, getpid, getppid, getuid, ioctl, keyctl, lseek, mkdirat, mknodat, mount, mprotect, munmap, newfstatat, openat2, pipe2, pivot_root, prctl, pread64, pselect6, readlink, readlinkat, rt_sigreturn, sched_yield, seccomp, set_robust_list, set_tid_address, setgid, setgroups, sethostname, setns, setresgid, setresuid, setsid, setuid, statfs, statx, symlinkat, tgkill, umask, umount2, unlinkat, unshare, write 10:08:25.616293 Wrote seccomp profile to: /tmp/profile.yaml 10:08:25.616298 Unloading bpf module I have to execute spoc as root because it will internally run an ebpf program by reusing the same code parts from the Security Profiles Operator itself. I can see that the bpf module got loaded successfully and spoc attached the required tracepoint to it. Then it will track the main application by using its mount namespace and process the recorded syscall data. The nature of ebpf programs is that they see the whole context of the Kernel, which means that spoc tracks all syscalls of the system, but does not interfere with their execution. The logs indicate that spoc found the syscalls read , close , mmap and so on, including uname . All other syscalls than uname are coming from the golang runtime and its garbage collection, which already adds overhead to a basic application like in our demo. I can also see from the log line Adding base syscalls: … that spoc adds a bunch of base syscalls to the resulting profile. Those are used by the OCI runtime (like runc or crun ) in order to be able to run a container. This means that spoc can be used to record seccomp profiles which then can be containerized directly. This behavior can be disabled in spoc by using the --no-base-syscalls /-n or customized via the --base-syscalls /-b command line flags This can be helpful in cases where different OCI runtimes other than crun and runc are used, or if I just want to record the seccomp profile for the application and stack it with another base profile . The resulting profile is now available in /tmp/profile.yaml , but the default location can be changed using the --output-file value /-o flag: cat /tmp/profile.yaml apiVersion : security-profiles-operator.x-k8s.io/v1beta1 kind : SeccompProfile metadata : creationTimestamp : null name : main spec : architectures : - SCMP_ARCH_X86_64 defaultAction : SCMP_ACT_ERRNO syscalls : - action : SCMP_ACT_ALLOW names : - access - arch_prctl - brk - … - uname - … status : {} The seccomp profile Custom Resource Definition (CRD) can be directly used together with the Security Profiles Operator for managing it within Kubernetes. spoc is also capable of producing raw seccomp profiles (as JSON), by using the --type /-t raw-seccomp flag: sudo ./spoc record --type raw-seccomp ./main … 52.628827 Wrote seccomp profile to: /tmp/profile.json jq . /tmp/profile.json { "defaultAction" : "SCMP_ACT_ERRNO" , "architectures" : ["SCMP_ARCH_X86_64" ], "syscalls" : [ { "names" : ["access" , "…" , "write" ], "action" : "SCMP_ACT_ALLOW" } ] } The utility spoc record allows us to record complex seccomp profiles directly from binary invocations in any Linux system which is capable of running the ebpf code within the Kernel. But it can do more: How about modifying the seccomp profile and then testing it by using spoc run . Running seccomp profiles with spoc run spoc is also able to run binaries with applied seccomp profiles, making it easy to test any modification to it. To do that, just run: sudo ./spoc run ./main 10:29:58.153263 Reading file /tmp/profile.yaml 10:29:58.153311 Assuming YAML profile 10:29:58.154138 Setting up seccomp 10:29:58.154178 Load seccomp profile 10:29:58.154189 Starting audit log enricher 10:29:58.154224 Enricher reading from file /var/log/audit/audit.log 10:29:58.155356 Running command with PID: 437880 It looks like that the application exited successfully, which is anticipated because I did not modify the previously recorded profile yet. I can also specify a custom location for the profile by using the --profile /-p flag, but this was not necessary because I did not modify the default output location from the record. spoc will automatically determine if it's a raw (JSON) or CRD (YAML) based seccomp profile and then apply it to the process. The Security Profiles Operator supports a log enricher feature , which provides additional seccomp related information by parsing the audit logs. spoc run uses the enricher in the same way to provide more data to the end users when it comes to debugging seccomp profiles. Now I have to modify the profile to see anything valuable in the output. For example, I could remove the allowed uname syscall: jq 'del(.syscalls[0].names[] | select(. == "uname"))' /tmp/profile.json /tmp/no-uname-profile.json And then try to run it again with the new profile /tmp/no-uname-profile.json : sudo ./spoc run -p /tmp/no-uname-profile.json ./main 10:39:12.707798 Reading file /tmp/no-uname-profile.json 10:39:12.707892 Setting up seccomp 10:39:12.707920 Load seccomp profile 10:39:12.707982 Starting audit log enricher 10:39:12.707998 Enricher reading from file /var/log/audit/audit.log 10:39:12.709164 Running command with PID: 480512 panic: operation not permitted goroutine 1 [running]: main.main() /path/to/main.go:10 +0x85 10:39:12.713035 Unable to run: launch runner: wait for command: exit status 2 Alright, that was expected! The applied seccomp profile blocks the uname syscall, which results in an "operation not permitted" error. This error is pretty generic and does not provide any hint on what got blocked by seccomp. It is generally extremely difficult to predict how applications behave if single syscalls are forbidden by seccomp. It could be possible that the application terminates like in our simple demo, but it could also lead to a strange misbehavior and the application does not stop at all. If I now change the default seccomp action of the profile from SCMP_ACT_ERRNO to SCMP_ACT_LOG like this: jq '.defaultAction = "SCMP_ACT_LOG"' /tmp/no-uname-profile.json /tmp/no-uname-profile-log.json Then the log enricher will give us a hint that the uname syscall got blocked when using spoc run : sudo ./spoc run -p /tmp/no-uname-profile-log.json ./main 10:48:07.470126 Reading file /tmp/no-uname-profile-log.json 10:48:07.470234 Setting up seccomp 10:48:07.470245 Load seccomp profile 10:48:07.470302 Starting audit log enricher 10:48:07.470339 Enricher reading from file /var/log/audit/audit.log 10:48:07.470889 Running command with PID: 522268 10:48:07.472007 Seccomp: uname (63) The application will not terminate any more, but seccomp will log the behavior to /var/log/audit/audit.log and spoc will parse the data to correlate it directly to our program. Generating the log messages to the audit subsystem comes with a large performance overhead and should be handled with care in production systems. It also comes with a security risk when running untrusted apps in audit mode in production environments. This demo should give you an impression how to debug seccomp profile issues with applications, probably by using our shiny new helper tool powered by the features of the Security Profiles Operator. spoc is a flexible and portable binary suitable for edge cases where resources are limited and even Kubern...

Microsoft employees, upset about not getting raises this year, voiced their displeasure on an internal chat platform and got a response from the company's marketing boss.

Amazon is rolling out a broader refresh of its Echo lineup.

No system prompt is safe. The system prompt is the initial set of instructions that sets the boundaries for an AI conversation. What rules the assistant should follow, what topics to avoid, how the assistant should format responses, and more. But users have found various workarounds to get the models to divulge their instructions. A list of notable system prompt leaks from Snap, Bing, ChatGPT, Perplexity AI, and GitHub Copilot Chat. Snap’s MyAI System Prompt (source) Pretend that you are havin

DevPod is infrastructure-independent and client-only, which makes it incredibly easy to get started with. Codespaces but open-source, client-only and unopinionated. Works with any infra, any progamming language, any IDE, etc.

Members of various community colleges tour and make plates in the Lurie Nanofabrication Facility on August 1, 2013. Image credit: Joseph Xu, Michigan Engineering Communications & Marketing On the heels of the global chip shortage, the University of Michigan is part of a new public-private partner

Sam Altman joined experts from IBM and New York University to ask Congress for swift AI regulation. They're just split on what form it should take.

The Bluesky Social application for Web, iOS, and Android

The U.S. has accused Temu of data risks following the Google suspension of its sister app, but analysts are not too bothered.

Brendan Burns, Kubernetes' co-founder shared his thoughts on GitOps and Kubernetes at GitOpsCon.

PipeWire 0.3.71 is out today as the newest update to this now widely-used open-source solution for managing Linux audio and video streams and serving as a viable replacement to the likes of PulseAudio and JACK for audio needs on the Linux desktop.

Did we forget about .pl, .sh and oh yeah, .com ?

Part of our work at Sternum includes constant security research of IoT vulnerabilities to better understand IoT security gaps, boost the security capabilities of our platform and help device manufacturers improve their security postures. In this post, we wanted to provide a behind-the-scenes look at our work and talk about our latest discovery—a buffer overflow […]

Following in the footsteps of VPNFilter, new firmware obscures hackers' endpoints.

A new division set up by the government to pursue sanctions evasion and technology espionage announced arrests of individuals with ties to foreign governments.

GitHub recently experienced several availability incidents, both long running and shorter duration. We have since mitigated these incidents and all systems are now operating normally. Read on for more details about what caused these incidents and what we’re doing to mitigate in the future.

Sign up for the GI Daily here to get the biggest news straight to your inbox Saudi Arabia's Public Investment Fund has …

Today’s guest is Dagna Bieda, a career coach who specializes in helping developers and engineers level up their careers. She shares why developers should promote the value of their contributions, how soft skills can make or break a coding career, and why a moment of burnout inspired her to start coaching.

Congress has resurrected the Kids Online Safety Act (KOSA), a bill that would increase surveillance and restrict access to information in the name of protecting children online. KOSA was introduced in 2022 but failed to gain traction, and today its authors, Sens. Richard Blumenthal (D-CT) and...

A researcher found that the Telegram app on macOS suffers from a flaw that could be exploited to gain unauthorized access to the camera.

This video provides an overview of how GitOps can help you enhance the security of your cloud native infrastructure and workloads. This video is based on the presentation I gave at ArgoCon: https://youtu.be/nGcvPAQdpVg 🎊 Here is the Demo repository on GitHub: https://github.com/Cloud-Native-Security/gitops-the-magickey 🎊 📚Additional Resources📚 * My weekly DevOps newsletter https://anaisurl.com/newsletter/ * My website https://anaisurl.com/ * Follow me on Twitter https://twitter.com/urlichsanais * GitHub https://github.com/AnaisUrlichs 🥰Ways to support my content work 🥰 * Please subscribe to my YouTube channel to support my content and give this video a Thumbs up if you enjoyed it. * If you want to support me further https://www.buymeacoffee.com/urlichsanais 📝Also, do comment on any topics that you would like to see covered in future videos. 🕒Timestamps🕒 00:00 - Intro 00:29 - Link to Recording 00:52 - GitOps best practices 01:38 - Software Supply Chain 03:28 - Demo Scanning ArgoCD Application Resources 11:28 - Outro

In this series of short stories, I share tricky situations I’ve encountered while leading teams. These experiences have taught me invaluable leadership lessons and greatly influenced my manag…

A provocative paper from researchers at Microsoft claims A.I. technology shows the ability to understand the way people do. Critics say those scientists are kidding themselves.

Developer News

Authors: Anish Ramasekar, Mo Khan, and Rita Zhang (Microsoft) With Kubernetes 1.27, we (SIG Auth) are moving Key Management Service (KMS) v2 API to beta. What is KMS? One of the first things to consider when securing a Kubernetes cluster is encrypting etcd data at rest. KMS provides an interface for a provider to utilize a key stored in an external key service to perform this encryption. KMS v1 has been a feature of Kubernetes since version 1.10, and is currently in beta as of version v1.12. KMS v2 was introduced as alpha in v1.25. Note The KMS v2 API and implementation changed in incompatible ways in-between the alpha release in v1.25 and the beta release in v1.27. The design of KMS v2 has changed since the previous blog post was written and it is not compatible with the design in this blog post. Attempting to upgrade from old versions with the alpha feature enabled will result in data loss. What’s new in v2beta1 ? The KMS encryption provider uses an envelope encryption scheme to encrypt data in etcd. The data is encrypted using a data encryption key (DEK). The DEKs are encrypted with a key encryption key (KEK) that is stored and managed in a remote KMS. With KMS v1, a new DEK is generated for each encryption. With KMS v2, a new DEK is only generated on server startup and when the KMS plugin informs the API server that a KEK rotation has occurred. Caution If you are running virtual machine (VM) based nodes that leverage VM state store with this feature, you must not use KMS v2. With KMS v2, the API server uses AES-GCM with a 12 byte nonce (8 byte atomic counter and 4 bytes random data) for encryption. The following issues could occur if the VM is saved and restored: The counter value may be lost or corrupted if the VM is saved in an inconsistent state or restored improperly. This can lead to a situation where the same counter value is used twice, resulting in the same nonce being used for two different messages. If the VM is restored to a previous state, the counter value may be set back to its previous value, resulting in the same nonce being used again. Although both of these cases are partially mitigated by the 4 byte random nonce, this can compromise the security of the encryption. Sequence Diagram Encrypt Request kube_api_server: create/update resource that's to be encrypted kube_api_server-kube_api_server: encrypt resource with DEK kube_api_server-etcd: store encrypted object ``` -- Decrypt Request kube_api_server: get/list resource that's encrypted kube_api_server-etcd: get encrypted resource etcd-kube_api_server: encrypted resource alt Encrypted DEK not in cache kube_api_server-kms_plugin: decrypt request kms_plugin-external_kms: decrypt DEK with remote KEK external_kms-kms_plugin: decrypted DEK kms_plugin-kube_api_server: return decrypted DEK kube_api_server-kube_api_server: cache decrypted DEK end kube_api_server-kube_api_server: decrypt resource with DEK kube_api_server-user: return decrypted resource ``` -- Status Request kms_plugin: status request kms_plugin-external_kms: validate remote KEK external_kms-kms_plugin: KEK status kms_plugin-kube_api_server: return status response {"healthz": "ok", key_id: "", "version": "v2beta1"} alt KEK rotation detected (key_id changed), rotate DEK Note over kube_api_server,external_kms: Refer to Generate Data Encryption Key (DEK) diagram for details end end ``` -- Generate Data Encryption Key (DEK) kube_api_server: generate DEK kube_api_server-kms_plugin: encrypt request kms_plugin-external_kms: encrypt DEK with remote KEK external_kms-kms_plugin: encrypted DEK kms_plugin-kube_api_server: return encrypt response {"ciphertext": "", key_id: "", "annotations": {}} ``` -- Performance Improvements With KMS v2, we have made significant improvements to the performance of the KMS encryption provider. In case of KMS v1, a new DEK is generated for every encryption. This means that for every write request, the API server makes a call to the KMS plugin to encrypt the DEK using the remote KEK. The API server also has to cache the DEKs to avoid making a call to the KMS plugin for every read request. When the API server restarts, it has to populate the cache by making a call to the KMS plugin for every DEK in the etcd store based on the cache size. This is a significant overhead for the API server. With KMS v2, the API server generates a DEK at startup and caches it. The API server also makes a call to the KMS plugin to encrypt the DEK using the remote KEK. This is a one-time call at startup and on KEK rotation. The API server then uses the cached DEK to encrypt the resources. This reduces the number of calls to the KMS plugin and improves the overall latency of the API server requests. We conducted a test that created 12k secrets and measured the time taken for the API server to encrypt the resources. The metric used was apiserver_storage_transformation_duration_seconds . For KMS v1, the test was run on a managed Kubernetes v1.25 cluster with 2 nodes. There was no additional load on the cluster during the test. For KMS v2, the test was run in the Kubernetes CI environment with the following cluster configuration . KMS Provider Time taken by 95 percentile KMS v1 160ms KMS v2 80μs The results show that the KMS v2 encryption provider is three orders of magnitude faster than the KMS v1 encryption provider. What's next? For Kubernetes v1.28, we expect the feature to stay in beta. In the coming releases we want to investigate: Cryptographic changes to remove the limitation on VM state store. Kubernetes REST API changes to enable a more robust story around key rotation. Handling undecryptable resources. Refer to the KEP for details. You can learn more about KMS v2 by reading Using a KMS provider for data encryption . You can also follow along on the KEP to track progress across the coming Kubernetes releases. Call to action In this blog post, we have covered the improvements made to the KMS encryption provider in Kubernetes v1.27. We have also discussed the new KMS v2 API and how it works. We would love to hear your feedback on this feature. In particular, we would like feedback from Kubernetes KMS plugin implementors as they go through the process of building their integrations with this new API. Please reach out to us on the #sig-auth-kms-dev channel on Kubernetes Slack. How to get involved If you are interested in getting involved in the development of this feature, share feedback, or participate in any other ongoing SIG Auth projects, please reach out on the #sig-auth channel on Kubernetes Slack. You are also welcome to join the bi-weekly SIG Auth meetings , held every-other Wednesday. Acknowledgements This feature has been an effort driven by contributors from several different companies. We would like to extend a huge thank you to everyone that contributed their time and effort to help make this possible.

YouTube TV is up 5 percent, and that's before NFL Sunday Ticket kicks in.

This blog post is written by Pranaya Anshu, EC2 PMM, and Sid Ambatipudi, EC2 Compute GTM Specialist. Amazon EC2 Spot Instances are a powerful tool that thousands of customers use to optimize their compute costs. The National Football League (NFL) is an example of customer using Spot Instances, leveraging 4000 EC2 Spot Instances across more […]