SYSTEM CALL

A syscall is the mechanism that allows a program running in user space to request services from the operating system kernel, such as reading from a file, writing to the console, or allocating memory. Instead of interacting directly with hardware, the program places specific values into designated registers to indicate the system call number and its arguments. Without syscalls, developers would need to manually communicate with hardware, for example writing to video memory and video I/O, handling character encoding, sending output to be printed, and waiting for confirmation of completion. Performing all these steps just to print a single character would make assembly code far longer and more complex, which is why syscalls provide a critical abstraction layer between software and hardware. On Linux x86 64, the system call number is placed in the rax register, while arguments are passed in rdi, rsi, rdx, r10, r8, and r9. Once the registers are prepared, the syscall instruction is executed, switching the processor into kernel mode and transferring control to the operating system. After the kernel completes the requested operation, it places the result or return value back into rax and returns execution to user space. Conceptually, a syscall can be thought of as a globally available function provided by the operating system kernel. It takes the required arguments via registers and executes the requested operation on behalf of the program. Linux syscalls and their corresponding numbers can be found in the system header file unistd_64.h.

root@dev:~$ cat /usr/include/x86_64-linux-gnu/asm/unistd_64.h
 #ifndef _ASM_X86_UNISTD_64_H
 #define _ASM_X86_UNISTD_64_H 1

 #define __NR_read 0
 #define __NR_write 1
 #define __NR_open 2
 #define __NR_close 3
 #define __NR_stat 4
 #define __NR_fstat 5

 * this file sets the syscall number for each syscall to refer to that syscall 
   using this number
    - With 32-bit x86 processors, the syscall numbers are in the unistd_32.h file.

REFERENCE: LINUX SYSCALLS (LOCAL)

//view how to use the write syscall
root@dev:~$ man -s 2 write
 ...SNIP...
 ssize_t write(int fd, const void *buf, size_t count);
 
 * the write syscall expects three arguments
    - File Descriptor fd to be printed to (usually 1 for stdout)
    - The address pointer to the string to be printed
    - The length to be printed
    
 * the -s 2 flag specifies syscall man pages.

REFERENCE: LINUX SYSCALL TABLE (ONLINE)

Searchable Linux Syscall Table for x86_64filippo.io
Logolinux/arch/x86/entry/syscalls/syscall_64.tbl at master · torvalds/linuxGitHub

SYSCALL CALLING CONVENTION

Save registers to the stack before any function call or syscall. However, if the call occurs at the start of the program when no registers hold meaningful values, saving them is not necessary.

All functions and syscalls should follow this standard and take their arguments from the corresponding registers.

DEBUGGING

GRACEFULLY EXITING SYSCALL

A graceful exit syscall allows a program to terminate cleanly by informing the operating system that it has completed execution and providing an optional exit code. In Linux x86-64 assembly, this is typically done using the exit syscall (number 60) with the return value placed in the rdi register. Failing to properly exit a program can lead to undefined behavior, including segmentation faults or other runtime errors, because the operating system may not correctly release resources or restore the process state. Using the exit syscall ensures that the program ends in a controlled manner, cleaning up resources, closing open file descriptors, and returning an appropriate status code to the operating system or calling process. This practice is essential in assembly programming, where there is no automatic runtime to manage process termination.

PreviousSTRINGNextWRITE

Last updated