fogserv.cloud fogserv.cloud

Python Basics — Python for Scripting and AI

Status: Active | Last Updated: 2026-09-15 Category: Languages — Python Prerequisites: Basic programming concepts (see kb/basics/) Tags: python, venv, pip, scripting, ai Estimated Time: 4-6 hours (Self-paced, includes lab time)

Summary

Python is the language of choice for AI/ML, scripting, and glue code. It uses indentation for block structure (no braces) and prioritizes readability. This article covers syntax, data structures, virtual environments, and the standard library modules you will use most often. By the end, you will be able to write, run, and distribute real Python scripts and understand why Python dominates the AI/ML ecosystem.

What You'll Learn (Core Competencies)

Table of Contents

  1. Architectural Overview & Core Schema
  2. Deep Dive & Implementation
  3. Anti-Patterns & Common Pitfalls
  4. Independent Challenge
  5. Consolidation & Key Invariants
  6. Next Steps

1. Architectural Overview & Core Schema

Why Python

Python is interpreted (no compile step), dynamically typed (variables have types but you do not declare them), and used heavily in:

The Interpretation Model

Python's execution model is distinct from compiled languages:

Layer What Exists Examples
Source Python with indentation def greet(name: str) -> str:
Compile-time Python compiler parses to bytecode .pyc files in __pycache__/
Runtime CPython VM executes bytecode Python integers are arbitrary-precision

Because Python is interpreted, there is no separate compilation step and no separate build artifact to ship. You ship the .py source (or bundle it with PyInstaller/Nuitka). The interpreter reads and executes source line by line, though CPython does compile to bytecode internally for performance.

Type Hints as Documentation and Tooling

Python 3.5 introduced type hints (PEP 484). They do not affect runtime behavior, but they are read by IDEs (autocomplete, error highlighting), static analyzers (mypy, pyright), and formatters (ruff). Type hints make code self-documenting and enable tooling that catches type mismatches before runtime.

def greet(name: str, count: int = 1) -> str:
    """Return a greeting string."""
    return ", ".join([f"Hello, {name}"] * count)

Without type hints, the function signature is opaque to tooling. With them, mypy can catch greet(42) before the code runs.

The GIL and Concurrency

Python has a Global Interpreter Lock (GIL), which allows only one thread to execute Python bytecode at a time. This simplifies memory management but limits CPU-bound parallelism. For I/O-bound tasks, asyncio (single-threaded async) or threading (useful for some I/O-bound tasks despite the GIL) are appropriate. For CPU-bound tasks, use multiprocessing (separate processes, each with its own GIL) or native C extensions. The GIL is irrelevant for I/O-bound workloads because the interpreter releases it during I/O waits.

2. Deep Dive & Implementation

Syntax Basics

Variables and Basic Types

Python variables spring into existence when you assign to them. There is no declaration step and no type annotation required, though type hints are available in Python 3.5+ for documentation and tooling. The interpreter infers the type from the right-hand side expression, and that type can change if you reassign the variable:

name = "fogserv"          # str
count = 42                # int
ratio = 3.14              # float
enabled = True            # bool
nothing = None            # NoneType (null-like)

Python integers have arbitrary precision, meaning count = 42 behaves identically to count = 10**100. Floats follow IEEE 754 double-precision, same as JavaScript's number type. Strings are immutable Unicode sequences. There is no separate character type — a single character is just a string of length 1.

Booleans are True and False (capitalized). Most objects have a truthiness: 0, "", [], {}, None evaluate to False; everything else is True. This makes conditionals concise:

items = []
if items:          # False — empty list
    print("has items")

Indentation as Block Structure

Python uses indentation instead of braces to delineate blocks. This is not a stylistic suggestion — the interpreter enforces it. A logical block (function body, if branch, class definition) is defined by consistent indentation. The official style guide (PEP 8) recommends 4 spaces per indentation level, and most tools enforce this. Mixing tabs and spaces causes an IndentationError in Python 3.

def classify(count: int) -> str:
    if count > 100:
        return "very many"
    elif count > 10:
        return "many"
    else:
        return "few"

The -> str and : int annotations above are type hints. They do not affect runtime behavior but are read by IDEs, mypy, and other static analysis tools. Always prefer 4 spaces. Never use tabs. Configure your editor to insert spaces when you press Tab.

Control Flow

Python's if/elif/else works as expected, but the elif keyword replaces the else if pattern from C/Java. The match/case statement (Python 3.10+) provides structural pattern matching:

status = "running"
match status:
    case "running":
        print("Service is up")
    case "stopped":
        print("Service is down")
    case _:
        print("Unknown status")

Loops use for and while. The for loop iterates over any iterable (list, string, file, range, etc.). The range() function generates sequences without materializing them in memory:

for i in range(10):      # 0 to 9
    print(i)

for char in "fogserv":
    print(char)

Use enumerate() when you need both index and value. Use zip() to iterate over multiple sequences in parallel:

servers = ["alpha", "beta", "gamma"]
ports   = [3000, 3001, 3002]

for i, (name, port) in enumerate(zip(servers, ports)):
    print(f"{i}: {name} on port {port}")

The break and continue statements work as in other languages. The else clause on loops runs only if the loop completes without hitting break — a useful pattern for search algorithms:

for n in range(2, 20):
    for d in range(2, n):
        if n % d == 0:
            break
    else:
        print(n, "is prime")

String Formatting

Python offers four ways to format strings. F-strings (formatted string literals, Python 3.6+) are the preferred modern style. They embed expressions inside {} with optional format specs:

name = "fogserv"
port = 3000
info = f"{name} running on port {port}"          # simple interpolation
width = f"{port:05d}"                            # zero-padded to 5 digits
price = f"${3.14159:.2f}"                        # 2 decimal places

The .format() method (Python 2.6+) uses positional or named placeholders. The %-formatting style (like sprintf) is the oldest and still common in legacy code:

"Hello, {}".format(name)         # .format() style
"Hello, %s" % name               # % style (legacy)

For machine-readable output (logging, APIs), prefer f-strings. For user-facing text with localization needs, use the gettext module or a higher-level i18n framework.

Data Structures

Lists

Lists are ordered, mutable sequences. They are the workhorse data structure in Python — used where arrays, stacks, queues, and dynamic arrays would be used in other languages. Lists are created with square brackets or the list() constructor:

stack = []
stack.append("first")
stack.append("second")
top = stack.pop()        # "second"

queue = []
queue.append("a")
queue.append("b")
first = queue.pop(0)     # "a"

List comprehensions provide a concise, Pythonic way to create lists from iterables. They combine a loop and element transformation in a single expression and are typically faster than equivalent for loops with .append():

squares = [x**2 for x in range(10)]           # [0, 1, 4, 9, 16, ...]
evens   = [x for x in range(20) if x % 2 == 0]
pairs   = [(a, b) for a in range(3) for b in range(3)]

Lists support slicing with list[start:stop:step]. Omitting a boundary means "from start" or "to end". Slicing returns a new list:

data = [0, 10, 20, 30, 40, 50]
data[1:4]     # [10, 20, 30]
data[:3]      # [0, 10, 20]
data[-2:]     # [40, 50]
data[::2]     # [0, 20, 40] — every other element
data[::-1]    # [50, 40, 30, 20, 10, 0] — reversed

Dictionaries

Dictionaries (dict) store key-value pairs with O(1) average-case lookup by hashable key. Keys must be immutable (str, int, tuple, etc.). Dicts preserve insertion order (since Python 3.7+, this is a language guarantee, not just an implementation detail):

config = {
    "host": "0.0.0.0",
    "port": 3000,
    "debug": False,
}

config["port"] = 8080           # update
config["timeout"] = 30          # insert
value = config.get("missing", "default")  # safe access with fallback

for key, value in config.items():
    print(f"{key} = {value}")

Dictionary comprehensions mirror list comprehensions:

squares = {k: k**2 for k in range(5)}   # {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}

Sets

Sets are unordered collections of unique, hashable elements. They support fast membership testing (O(1)), union, intersection, and difference operations. Use sets when you need to track unique values or test set membership:

tags = {"python", "cli", "automation"}
tags.add("scripting")
tags.remove("cli")

if "python" in tags:
    print("Python is in the set")

a = {1, 2, 3, 4}
b = {3, 4, 5, 6}
print(a & b)   # {3, 4} — intersection
print(a | b)    # {1, 2, 3, 4, 5, 6} — union
print(a - b)    # {1, 2} — difference

Tuples

Tuples are immutable, ordered sequences. They serve two main roles: as fixed-length records (like lightweight structs) and as dict keys (since immutable). Unlike lists, tuples are hashable if all their elements are hashable:

point = (10, 20)
x, y = point          # unpacking

# Named tuple for clearer record access
from collections import namedtuple
Server = namedtuple("Server", ["name", "host", "port"])
srv = Server("prod-1", "10.0.0.5", 3000)
print(srv.name)        # "prod-1"

Because tuples are immutable, they can serve as dictionary keys or be placed in sets, whereas lists cannot. Use tuples for fixed-length heterogeneous sequences and lists for variable-length homogeneous ones.

When to Use Which

Choose based on your access pattern: indexed by position (list/tuple), indexed by key (dict), unique membership (set), fixed record (tuple or namedtuple). A common beginner mistake is using a list where a dict or set is more appropriate. If you find yourself writing for item in items: if item.key == value, consider a dict keyed by item.key instead.

Functions and Classes

Defining and Calling Functions

Functions are defined with def, can accept positional and keyword arguments, and return values with return. Python supports default argument values, *args for variadic positionals, and **kwargs for variadic keyword arguments:

def greet(name: str, greeting: str = "Hello") -> str:
    """Return a greeting string. Docstrings go triple-quoted."""
    return f"{greeting}, {name}!"

def fetch_all(*urls: str) -> list[str]:
    results = []
    for url in urls:
        results.append(fetch_one(url))
    return results

def configure(**settings: str | int) -> dict:
    return {k.upper(): v for k, v in settings.items()}

Python functions are first-class objects: they can be assigned to variables, passed as arguments, and returned from other functions. This enables powerful functional programming patterns:

numbers = [3, 1, 4, 1, 5, 9, 2, 6]
sorted_nums = sorted(numbers, key=lambda x: -x)   # descending

Classes and Objects

Python's class system is single-inheritance with multiple interface inheritance through Abstract Base Classes (ABCs). The __init__ method is the initializer (not the constructor — __new__ handles that). Instance variables are created by assignment in __init__ or any method:

class Server:
    def __init__(self, name: str, host: str, port: int):
        self.name = name          # public attribute
        self._host = host         # convention: "protected"
        self.__port = port        # name-mangled to _Server__port

    @property
    def address(self) -> str:
        return f"{self._host}:{self.__port}"

    def start(self) -> None:
        print(f"Starting {self.name} at {self.address}")

srv = Server("prod", "0.0.0.0", 3000)
srv.start()
print(srv.address)      # calls the property method

Methods marked with @property look like attribute access but execute code. This is the Pythonic way to expose computed attributes. Use @staticmethod for methods that do not need self, and @classmethod when a method needs to create an instance via an alternate constructor:

class Config:
    def __init__(self, data: dict):
        self._data = data

    @classmethod
    def from_file(cls, path: str) -> "Config":
        import json
        with open(path) as f:
            return cls(json.load(f))

Inheritance and MRO

Python supports single inheritance and multiple inheritance via the Method Resolution Order (MRO), which follows C3 linearization. Use super() to call the parent class method, which correctly traverses the MRO:

class BaseServer:
    def start(self) -> None:
        print("Starting base")

class HTTPServer(BaseServer):
    def start(self) -> None:
        print("Starting HTTP server")
        super().start()          # call parent's start()

Modules and Imports

Every .py file is a module. Packages are directories with an __init__.py. Import with import module or from module import name. Avoid from module import * — it pollutes the namespace. Use absolute imports (from the project root) or explicit relative imports:

# absolute import
from kb.runtime import bun_runtime

# relative import (inside a package)
from . import utils
from ..languages import python_basics

The __all__ list in a module controls what from module import * exports. Always define __all__ if you expect others to use star imports.

Virtual Environments

Why Virtual Environments

Python's global site-packages directory is shared across all projects. Installing a library for one project can silently break another project that depends on a different version of the same library. Virtual environments solve this by creating isolated Python installations, each with their own site-packages directory.

Creating and Managing venv

The venv module is part of the standard library (Python 3.3+). Create an environment with:

# Create a virtual environment named .venv in the current directory
python3 -m venv .venv

# Activate it (Linux/macOS)
source .venv/bin/activate

# Activate it (Windows PowerShell)
.venv\Scripts\Activate.ps1

# Now your prompt shows (.venv) and pip points to the isolated env
pip install requests httpx

When the environment is active, pip install installs into .venv/lib/python3.x/site-packages/, not the global site-packages. Deactivate with the deactivate command.

pip and Requirements Files

pip is the package installer. It downloads packages from PyPI (Python Package Index) by default. Pin your dependencies to specific versions to ensure reproducible builds:

pip freeze > requirements.txt      # export current env's packages
pip install -r requirements.txt   # install from file

The output of pip freeze includes every package, including transitive dependencies. For cleaner dependency files, consider pip-tools with its pip-compile command, which resolves dependencies and produces a locked requirements file:

# requirements.in — human-written, loose deps
# package>=1.0
# another-package

pip-compile requirements.in        # produces requirements.txt with pinned versions

pyproject.toml (Modern Standard)

Modern Python projects use pyproject.toml (PEP 517/518) instead of setup.py or requirements.txt for build system configuration. The hatch, poetry, and pip-tools ecosystems all respect this format:

[project]
name = "fogserv-agent"
version = "0.1.0"
dependencies = [
    "httpx>=0.27.0",
    "pydantic>=2.0",
]

[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff>=0.5"]

With this setup, pip install -e ".[dev]" installs the package in editable mode with dev dependencies. Use ruff or black for formatting and mypy or pyright for type checking in CI.

Common Stdlib

os — Operating System Interface

The os module provides portable access to OS services. Always prefer pathlib.Path for path manipulation over string concatenation:

import os
from pathlib import Path

# Legacy os.path style
config_path = os.path.join(os.path.expanduser("~"), ".config", "app")

# Modern pathlib style
config_dir = Path.home() / ".config" / "app"
config_dir.mkdir(parents=True, exist_ok=True)
(config_dir / "settings.json").write_text(json.dumps(settings))

Walk a directory tree with os.walk or the Path.rglob() method for glob-based traversal:

for py_file in Path("kb").rglob("*.md"):
    print(py_file)

json — JSON Encoding and Decoding

JSON is the lingua franca of web APIs. Python's json module serializes Python objects to JSON strings and back:

import json

data = {"name": "fogserv", "ports": [3000, 3001]}
json_str = json.dumps(data, indent=2)       # dict → JSON string
parsed   = json.loads(json_str)             # JSON string → dict

# Work with files directly
with open("config.json") as f:
    config = json.load(f)                   # file → dict

with open("out.json", "w") as f:
    json.dump(config, f, indent=2)          # dict → file

For large JSON files, use ijson (streaming parser) or orjson (faster C-based encoder).

argparse — Command-Line Argument Parsing

For CLI scripts, argparse builds structured argument parsers with help text, type coercion, and subcommand support:

import argparse

parser = argparse.ArgumentParser(description="Deploy a service")
parser.add_argument("name", help="Service name")
parser.add_argument("--port", type=int, default=3000, help="Port number")
parser.add_argument("--verbose", "-v", action="store_true")
args = parser.parse_args()

if args.verbose:
    print(f"Deploying {args.name} on port {args.port}")

Use argparse for any script that accepts arguments. For complex CLIs with subcommands, use click or typer instead — they build on top of argparse but offer a decorator-based interface.

datetime — Dates and Times

Python's datetime module handles dates, times, and their combinations. Always use timezone-aware datetimes in production:

from datetime import datetime, timezone

# UTC-aware current time
now = datetime.now(timezone.utc)

# Parse ISO 8601 string
timestamp = datetime.fromisoformat("2026-09-15T10:30:00Z")

# Format for display or storage
print(now.strftime("%Y-%m-%d %H:%M:%S %Z"))

For timezone conversions, use zoneinfo.ZoneInfo (Python 3.9+). For date arithmetic, use timedelta. Avoid naive datetimes (without timezone) in server code — they are a common source of off-by-one bugs in production.

pathlib — Object-Oriented Filesystem Paths

pathlib (Python 3.4+) replaces os.path and os file functions with an object-oriented API. Path objects represent filesystem paths and expose methods for reading, writing, globbing, and directory traversal:

from pathlib import Path

p = Path("kb/runtime")
p.exists()           # True if path exists
p.is_dir()           # True if directory
list(p.glob("*.md")) # all .md files in runtime/

# Read/write files
content = Path("README.md").read_text()
Path("out.txt").write_text("hello")

Prefer pathlib over open() for simple reads/writes and over os.path.join for path construction. It is more readable and composable.

subprocess — Running External Commands

For running shell commands from Python, use subprocess.run() (Python 3.5+). Avoid os.system() and subprocess.call() — they are older and less flexible:

import subprocess

result = subprocess.run(
    ["git", "log", "--oneline", "-5"],
    capture_output=True,
    text=True,
    check=True,              # raise CalledProcessError on non-zero exit
)
print(result.stdout)

For long-running processes, use subprocess.Popen for streaming I/O or the asyncio module with asyncio.create_subprocess_exec for async execution.

Guided Checkpoint

Create a virtual environment, install a package, and run a script that uses argparse, pathlib, and datetime:

# Create a fresh venv and install httpx
python3 -m venv /tmp/py-checkpoint
source /tmp/py-checkpoint/bin/activate
pip install httpx --quiet

# Write and run a small CLI script
cat > /tmp/py-checkpoint/fetch.py <<'EOF'
import argparse, sys
from pathlib import Path
from datetime import datetime, timezone

parser = argparse.ArgumentParser(description="Fetch a URL and log metadata")
parser.add_argument("url", help="URL to fetch")
parser.add_argument("--out", default="response.txt", help="Output file")
args = parser.parse_args()

import httpx
with httpx.Client() as client:
    r = client.get(args.url)

Path(args.out).write_text(f"[{datetime.now(timezone.utc).isoformat()}] {r.status_code} {len(r.content)} bytes\n")
print(f"Wrote {Path(args.out).stat().st_size} bytes to {args.out}")
EOF

# Run with a valid URL
python /tmp/py-checkpoint/fetch.py https://httpbin.org/get --out /tmp/py-checkpoint/response.txt
# Expected: writes response metadata to response.txt, no exceptions

deactivate

3. Anti-Patterns & Common Pitfalls

These failure modes appear repeatedly in Python codebases. Each is detectable, preventable, and often rooted in Python's dynamic nature.

Anti-Pattern: Using a List Where a Dict or Set Is Appropriate

A common beginner mistake is iterating over a list to find an element, when a dict keyed by that element is O(1) instead of O(n).

# Bad: O(n) lookup every time
users = [{"id": 1, "name": "alice"}, {"id": 2, "name": "bob"}]
for u in users:
    if u["name"] == "alice":
        return u

# Good: O(1) lookup by id
users_by_name = {u["name"]: u for u in users}
return users_by_name["alice"]

Detection: search for for ... in loops with if ... == that could be keyed lookups.

Fix: understand the access pattern before choosing a data structure. Unique membership → set. Keyed lookup → dict. Ordered sequence with index → list.

Anti-Pattern: Modifying a List While Iterating Over It

Python's for loop uses the iterator directly; modifying the underlying list invalidates it, causing skipped elements or unexpected behavior.

# Bad: modifies list while iterating — elements are skipped
items = [1, 2, 3, 4, 5]
for i in items:
    if i % 2 == 0:
        items.remove(i)  # may skip the element after the removed one

# Good: iterate over a copy
items = [1, 2, 3, 4, 5]
for i in list(items):
    if i % 2 == 0:
        items.remove(i)

# Best: use a list comprehension
items = [x for x in items if x % 2 != 0]

Detection: for x in list: ... list.remove(x) or list.pop(index) inside a loop.

Fix: prefer a list comprehension to filter in one pass, or build a new list and replace the original.

Anti-Pattern: Mutable Default Arguments

Default argument values are evaluated once at function definition time, not at each call. A mutable default argument is shared across all calls.

# Bad: the list grows across every call
def add_item(item, items=[]):
    items.append(item)
    return items

print(add_item("a"))  # ['a']
print(add_item("b"))  # ['a', 'b']  — "b" is appended to the shared list

# Good: use None and create the list inside the function
def add_item(item, items=None):
    if items is None:
        items = []
    items.append(item)
    return items

print(add_item("a"))  # ['a']
print(add_item("b"))  # ['b']

Detection: def f(..., items=[]) or any mutable object as a default argument.

Fix: use None as the default and initialize inside the function body.

Anti-Pattern: Naive Datetimes in Production

Datetime objects without timezone information are "naive." They do not carry context about what time zone they represent, making arithmetic and serialization ambiguous.

# Bad: naive datetime — what timezone is "local"?
import datetime
now = datetime.datetime.now()  # naive

# Good: always use timezone-aware datetimes
from datetime import datetime, timezone
now = datetime.now(timezone.utc)  # always UTC, always explicit

Detection: datetime.now() without an argument or datetime.now(timezone.utc) not used in date handling code.

Fix: configure all date sources to emit UTC-aware datetimes. Use zoneinfo.ZoneInfo for conversions. Set a policy: no naive datetimes in server code.

Anti-Pattern: Using Bare except: Without Specifying Exceptions

A bare except: catches KeyboardInterrupt, SystemExit, and every other BaseException subclass, making graceful shutdown impossible.

# Bad: catches everything including KeyboardInterrupt
try:
    risky_operation()
except:
    log_error()

# Good: catch only the exceptions you can handle
try:
    risky_operation()
except (ValueError, IOError) as e:
    log_error(e)
    raise  # re-raise if you cannot fully handle it

Detection: except: without a type, or catching Exception when a specific type is appropriate.

Fix: catch the narrowest possible exception type. Use except Exception only when you genuinely want to handle all non-system-exit errors.

4. Independent Challenge

Build a small log-file analysis CLI without step-by-step guidance.

Produce a single Python script (targeting Python 3.10+) that:

  1. Accepts a path to a log file as a positional argument and an optional --level filter (DEBUG, INFO, WARNING, ERROR, CRITICAL).
  2. Parses each line assuming a format like [2026-09-15T10:30:00] [INFO] Message here — a timestamp in brackets, a level in brackets, and the rest as the message.
  3. Aggregates and prints:
    • Total line count and count per level.
    • The most common message prefix (e.g., the first 50 characters before a colon or the first word).
    • A deduplicated list of all unique messages at or above the specified level.
  4. Writes the deduplicated messages to a file at the same directory as the input with _filtered.txt appended.

Constraints:

Deliverable: a single .py file. Verify by feeding it a generated log file. No solution is provided.

5. Consolidation & Key Invariants

6. Next Steps

With Python fundamentals in place, continue your learning path:

Change Log

Date Change
2026-09-15 Full hydration: expanded all sections with substantive content, added code examples for every concept, added Next Steps and Change Log
2026-08-29 Initial stub created, section headers defined
2026-09-15 Migrated to canonical 6-section template (Architectural Overview, Deep Dive with Guided Checkpoint, Anti-Patterns, Independent Challenge, Consolidation, Next Steps). Preserved all prior content, prerequisites, and change log entries with dates.

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse