Python File Handling & with open()
In computer operating systems, persistent files are stored on disk (SSD/HDD). When Python interacts with a file, the OS creates an open File Descriptor and returns a stream wrapper object in memory.
The built-in open(file, mode="r", encoding="utf-8") function connects Python to the physical file system.
Comprehensive Python File Modes:
| Mode | Name | Behavior / Stream Position | File Must Exist? |
|---|---|---|---|
'r' | Read (Default) | Opens file for reading text. Pointer placed at start. | Yes (raises FileNotFoundError if missing) |
'w' | Write (Overwrite) | Truncates (wipes) file to 0 bytes and writes from start. | No (Creates file if missing) |
'a' | Append | Writes new data to the end of file without erasing existing content. | No (Creates file if missing) |
'x' | Exclusive Creation | Creates and opens file for writing. Fails if file already exists! | Must NOT exist (raises FileExistsError) |
'r+' | Read & Write | Opens for bidirectional reading and writing. | Yes |
'rb' / 'wb' | Binary Modes | Reads/writes raw bytes (images, PDFs, audio, pickles). | Matches 'r' / 'w' rules |
# Writing and Reading using the open() function:
# Step 1: Open file in write mode ('w') with explicit UTF-8 encoding
f_write = open("sample_demo.txt", "w", encoding="utf-8")
f_write.write("Line 1: Hello from Python File I/O! 🚀\n")
f_write.write("Line 2: Built-in file streaming is fast and portable.\n")
f_write.close() # Always close manually if not using 'with'!
# Step 2: Open file in read mode ('r')
f_read = open("sample_demo.txt", "r", encoding="utf-8")
file_contents = f_read.read()
f_read.close()
print("--- File Contents Read From Disk ---")
print(file_contents)If you omit encoding="utf-8", Windows defaults to legacy cp1252 or ANSI, causing fatal UnicodeDecodeError crashes when reading emojis, non-English names, or foreign currency symbols!
In the manual open() ... close() pattern, if an unexpected exception occurs between open() and close(), the file remains locked in OS memory, causing resource leaks and file locking crashes.
The with open(...) as f: statement implements the Python Context Manager protocol (__enter__ and __exit__). It guarantees that the file stream is 100% automatically flushed and closed the microsecond execution exits the block, even if an unhandled exception or return statement occurs!
# Safe, industry-standard file handling with Context Manager:
with open("audit_log.txt", "w", encoding="utf-8") as file:
file.write("2026-08-14 10:00:00 [INFO] System initialized successfully.\n")
file.write("2026-08-14 10:05:22 [INFO] Database connection active.\n")
# Verify that Python automatically closed the file descriptor:
print("Is the file closed?", file.closed) # True! Guaranteed!file.closed evaluates to True immediately outside the with block. You never need to call file.close() manually ever again.
Python provides three different techniques for reading text streams:
f.read(): Reads the entire file into a single string in RAM. (Good for small config files; dangerous for 10 GB log files!).f.readline(): Reads a single line up to the newline\ncharacter.f.readlines(): Reads all lines and returns them as a Python list of strings.- Memory-Efficient Line Iterator (
for line in f:): Streams lines one by one from the OS buffer in $O(1)$ memory! This is the gold standard for large datasets.
# 1. Prepare multi-line test file:
with open("server_metrics.txt", "w", encoding="utf-8") as f:
f.write("CPU_Usage: 24%\nMemory_Usage: 58%\nDisk_Free: 412GB\nActive_Threads: 16\n")
# 2. Strategy A: Memory-Efficient Line-by-Line Streaming (O(1) RAM):
print("--- 🚀 Strategy A: Streaming lines with for loop ---")
with open("server_metrics.txt", "r", encoding="utf-8") as f:
for line_num, line in enumerate(f, start=1):
# line.strip() removes trailing newline character:
print(f"Line {line_num}: {line.strip()}")
# 3. Strategy B: readlines() returning a list:
with open("server_metrics.txt", "r", encoding="utf-8") as f:
all_lines = f.readlines()
print(f"\nTotal Lines in list: {len(all_lines)}")When processing massive multi-gigabyte log files, NEVER use f.read() or f.readlines(). Always stream lines using for line in file_handle: to maintain a negligible memory footprint.
Opening a file in write mode ("w") immediately erases all prior content. To preserve existing data and add new records to the bottom, open the file in append mode ("a"):
# 1. Initialize log file with header in 'w' mode:
with open("app_activity.log", "w", encoding="utf-8") as log_file:
log_file.write("=== APPLICATION EVENT LOG ===\n")
# 2. Append new user actions dynamically using 'a' mode:
def append_log_event(event_message):
with open("app_activity.log", "a", encoding="utf-8") as log_file:
log_file.write(f"• EVENT: {event_message}\n")
append_log_event("User 'balaji' logged into the portal")
append_log_event("Payment of Rs.1499 processed successfully")
append_log_event("User downloaded invoice PDF")
# Read back accumulated log:
with open("app_activity.log", "r", encoding="utf-8") as log_file:
print(log_file.read())In "a" mode, the OS file pointer is automatically moved to the end of the file before every write operation.
Opening an existing file in "w" mode immediately wipes and truncates its entire content to 0 bytes before you even write a single character. Always use "a" mode when adding records to existing files.
Write a small program that asks for a grocery item, appends it to "groceries.txt", and then reads back all saved items with line numbers.
# Writing items to shopping list:
items = ["Milk", "Whole Wheat Bread", "Eggs", "Green Tea"]
with open("groceries.txt", "w", encoding="utf-8") as f:
for item in items:
f.write(f"{item}\n")
print("🛒 Saved Grocery List:")
with open("groceries.txt", "r", encoding="utf-8") as f:
for idx, line in enumerate(f, 1):
print(f"{idx}. {line.strip()}")Q Why does Python not close files immediately when leaving a regular function without "with"?
CPython uses reference counting to close files when the file variable goes out of scope, but garbage collection timing is not guaranteed in other Python implementations (PyPy, Jython). with open() guarantees immediate deterministic closure on all platforms.
Q What is the difference between text mode and binary mode?
Text mode ("r", "w") handles string encoding (UTF-8) and normalizes platform-specific line endings (\r\n on Windows to \n in Python). Binary mode ("rb", "wb") reads/writes raw unencoded bytes directly.
Q How do I check if a file is already open or closed in Python?
Inspect the boolean attribute file_handle.closed. It returns True if closed and False if still open.