Linux Device Drivers
Jonathan Corbet, Alessandro Rubini,
and Greg Kroah-Hartman.
Web page format by http://linux.coconia.net
If you redistribute this page you must not remove the attribution to the original authors, Corbet, Rubini and Kroah-Hartman, or, to linux.coconia.net.
Index
Numbers
A B C D E F G I H J K L M
N O P Q R S T U V W X Z
Chapters
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Preface
Jon's Introduction
Alessandro's Introduction
Greg's Introduction
Audience for This Book
Organization of the Material
Background Information
Online Version and License
Conventions Used in This Book
Using Code Examples
We'd Like to Hear from You
Safari Enabled
Acknowledgments
Jon
Alessandro
Greg
Chapter 1
An Introduction to Device Drivers
The Role of the Device Driver
Splitting the Kernel
Loadable Modules
Classes of Devices and Modules
Security Issues
Version Numbering
License Terms
Joining the Kernel Development Community
Overview of the Book
Chapter 2
Building and Running Modules
Setting Up Your Test System
The Hello World Module
Kernel Modules Versus Applications
User Space and Kernel Space
Concurrency in the Kernel
The Current Process
A Few Other Details
Compiling and Loading
Compiling Modules
Loading and Unloading Modules
Version Dependency
Platform Dependency
The Kernel Symbol Table
Preliminaries
Initialization and Shutdown
The Cleanup Function
Error Handling During Initialization
Module-Loading Races
Module Parameters
Doing It in User Space
Quick Reference
Chapter 3
Char Drivers
The Design of scull
Major and Minor Numbers
The Internal Representation of Device Numbers
Allocating and Freeing Device Numbers
Dynamic Allocation of Major Numbers
Some Important Data Structures
File Operations
The file Structure
The inode Structure
Char Device Registration
Device Registration in scull
The Older Way
open and release
The open Method
The release Method
scull's Memory Usage
read and write
The read Method
The write Method
readv and writev
Playing with the New Devices
Quick Reference
Chapter 4
Debugging Techniques
Debugging Support in the Kernel
Debugging by Printing
printk
Redirecting Console Messages
How Messages Get Logged
Turning the Messages On and Off
Rate Limiting
Printing Device Numbers
Debugging by Querying
Using the /proc Filesystem
Implementing files in /proc
An older interface
Creating your /proc file
The seq_file interface
The ioctl Method
Debugging by Watching
Debugging System Faults
Oops Messages
System Hangs
Debuggers and Related Tools
Using gdb
The kdb Kernel Debugger
The kgdb Patches
The User-Mode Linux Port
The Linux Trace Toolkit
Dynamic Probes
Chapter 5
Concurrency and Race Conditions
Pitfalls in scull
Concurrency and Its Management
Semaphores and Mutexes
The Linux Semaphore Implementation
Using Semaphores in scull
Reader/Writer Semaphores
Completions
Spinlocks
Introduction to the Spinlock API
Spinlocks and Atomic Context
The Spinlock Functions
Reader/Writer Spinlocks
Locking Traps
Ambiguous Rules
Lock Ordering Rules
Fine- Versus Coarse-Grained Locking
Alternatives to Locking
Lock-Free Algorithms
Atomic Variables
Bit Operations
seqlocks
Read-Copy-Update
Quick Reference
Chapter 6
Advanced Char Driver Operations
ioctl
Choosing the ioctl Commands
The Return Value
The Predefined Commands
Using the ioctl Argument
Capabilities and Restricted Operations
The Implementation of the ioctl Commands
Device Control Without ioctl
Blocking I/O
Introduction to Sleeping
Simple Sleeping
Blocking and Nonblocking Operations
A Blocking I/O Example
Advanced Sleeping
How a process sleeps
Manual sleeps
Exclusive waits
The details of waking up
Ancient history: sleep_on
Testing the Scullpipe Driver
poll and select
Interaction with read and write
Reading data from the device
Writing to the device
Flushing pending output
The Underlying Data Structure
Asynchronous Notification
The Driver's Point of View
Seeking a Device
The llseek Implementation
Access Control on a Device File
Single-Open Devices
Restricting Access to a Single User at a Time
Blocking open as an Alternative to EBUSY
Cloning the Device on open
Quick Reference
Chapter 7
Time, Delays, and Deferred Work
Measuring Time Lapses
Using the jiffies Counter
Processor-Specific Registers
Knowing the Current Time
Delaying Execution
Long Delays
Busy waiting
Yielding the processor
Timeouts
Short Delays
Kernel Timers
The Timer API
The Implementation of Kernel Timers
Tasklets
Workqueues
The Shared Queue
Quick Reference
Timekeeping
Delays
Kernel Timers
Tasklets
Workqueues
Chapter 8
Allocating Memory
The Real Story of kmalloc
The Flags Argument
Memory zones
The Size Argument
Lookaside Caches
A scull Based on the Slab Caches: scullc
Memory Pools
get_free_page and Friends
A scull Using Whole Pages: scullp
The alloc_pages Interface
vmalloc and Friends
A scull Using Virtual Addresses: scullv
Per-CPU Variables
Obtaining Large Buffers
Acquiring a Dedicated Buffer at Boot Time
Quick Reference
Chapter 9
Communicating with Hardware
I/O Ports and I/O Memory
I/O Registers and Conventional Memory
Using I/O Ports
I/O Port Allocation
Manipulating I/O ports
I/O Port Access from User Space
String Operations
Pausing I/O
Platform Dependencies
An I/O Port Example
An Overview of the Parallel Port
A Sample Driver
Using I/O Memory
I/O Memory Allocation and Mapping
Accessing I/O Memory
Ports as I/O Memory
Reusing short for I/O Memory
ISA Memory Below 1 MB
isa_readb and Friends
Quick Reference
Chapter 10
Interrupt Handling
Preparing the Parallel Port
Installing an Interrupt Handler
The /proc Interface
Auto-Detecting the IRQ Number
Kernel-assisted probing
Do-it-yourself probing
Fast and Slow Handlers
The internals of interrupt handling on the x86
Implementing a Handler
Handler Arguments and Return Value
Enabling and Disabling Interrupts
Disabling a single interrupt
Disabling all interrupts
Top and Bottom Halves
Tasklets
Workqueues
Interrupt Sharing
Installing a Shared Handler
Running the Handler
The /proc Interface and Shared Interrupts
Interrupt-Driven I/O
A Write-Buffering Example
Quick Reference
|
Chapter 11
Data Types in the Kernel
Use of Standard C Types
Assigning an Explicit Size to Data Items
Interface-Specific Types
Other Portability Issues
Time Intervals
Page Size
Byte Order
Data Alignment
Pointers and Error Values
Linked Lists
Quick Reference
Chapter 12
PCI Drivers
The PCI Interface
PCI Addressing
Boot Time
Configuration Registers and Initialization
MODULE_DEVICE_TABLE
Registering a PCI Driver
Old-Style PCI Probing
Enabling the PCI Device
Accessing the Configuration Space
Accessing the I/O and Memory Spaces
PCI Interrupts
Hardware Abstractions
A Look Back: ISA
Hardware Resources
ISA Programming
The Plug-and-Play Specification
PC/104 and PC/104+
Other PC Buses
MCA
EISA
VLB
SBus
NuBus
External Buses
Quick Reference
Chapter 13
USB Drivers
USB Device Basics
Endpoints
Interfaces
Configurations
USB and Sysfs
USB Urbs
struct urb
Creating and Destroying Urbs
Interrupt urbs
Bulk urbs
Control urbs
Isochronous urbs
Submitting Urbs
Completing Urbs: The Completion Callback Handler
Canceling Urbs
Writing a USB Driver
What Devices Does the Driver Support?
Registering a USB Driver
probe and disconnect in Detail
Submitting and Controlling a Urb
USB Transfers Without Urbs
usb_bulk_msg
usb_control_msg
Other USB Data Functions
Quick Reference
Chapter 14
The Linux Device Model
Kobjects, Ksets, and Subsystems
Kobject Basics
Embedding kobjects
Kobject initialization
Reference count manipulation
Release functions and kobject types
Kobject Hierarchies, Ksets, and Subsystems
Ksets
Operations on ksets
Subsystems
Low-Level Sysfs Operations
Default Attributes
Nondefault Attributes
Binary Attributes
Symbolic Links
Hotplug Event Generation
Hotplug Operations
Buses, Devices, and Drivers
Buses
Bus registration
Bus methods
Iterating over devices and drivers
Bus attributes
Devices
Device registration
Device attributes
Device structure embedding
Device Drivers
Driver structure embedding
Classes
The class_simple Interface
The Full Class Interface
Managing classes
Class devices
Class interfaces
Putting It All Together
Add a Device
Remove a Device
Add a Driver
Remove a Driver
Hotplug
Dynamic Devices
The /sbin/hotplug Utility
IEEE1394 (FireWire)
Networking
PCI
Input
USB
SCSI
Laptop docking stations
S/390 and zSeries
Using /sbin/hotplug
Linux hotplug scripts
udev
Dealing with Firmware
The Kernel Firmware Interface
How It Works
Quick Reference
Kobjects
Sysfs Operations
Buses, Devices, and Drivers
Classes
Firmware
Chapter 15
Memory Mapping and DMA
Memory Management in Linux
Address Types
Physical Addresses and Pages
High and Low Memory
The Memory Map and Struct Page
Page Tables
Virtual Memory Areas
The vm_area_struct structure
The Process Memory Map
The mmap Device Operation
Using remap_pfn_range
A Simple Implementation
Adding VMA Operations
Mapping Memory with nopage
Remapping Specific I/O Regions
Remapping RAM
Remapping RAM with the nopage method
Remapping Kernel Virtual Addresses
Performing Direct I/O
Asynchronous I/O
An asynchronous I/O example
Direct Memory Access
Overview of a DMA Data Transfer
Allocating the DMA Buffer
Do-it-yourself allocation
Bus Addresses
The Generic DMA Layer
Dealing with difficult hardware
DMA mappings
Setting up coherent DMA mappings
DMA pools
Setting up streaming DMA mappings
Single-page streaming mappings
Scatter/gather mappings
PCI double-address cycle mappings
A simple PCI DMA example
DMA for ISA Devices
Registering DMA usage
Talking to the DMA controller
Quick Reference
Introductory Material
Implementing mmap
Implementing Direct I/O
Direct Memory Access
Chapter 16
Block Drivers
Registration
Block Driver Registration
Disk Registration
Block device operations
The gendisk structure
Initialization in sbull
A Note on Sector Sizes
The Block Device Operations
The open and release Methods
Supporting Removable Media
The ioctl Method
Request Processing
Introduction to the request Method
A Simple request Method
Request Queues
Queue creation and deletion
Queueing functions
Queue control functions
The Anatomy of a Request
The bio structure
request structure fields
Barrier requests
Nonretryable requests
Request Completion Functions
Working with bios
Block requests and DMA
Doing without a request queue
Some Other Details
Command Pre-Preparation
Tagged Command Queueing
Quick Reference
Chapter 17
Network Drivers
How snull Is Designed
Assigning IP Numbers
The Physical Transport of Packets
Connecting to the Kernel
Device Registration
Initializing Each Device
Module Unloading
The net_device Structure in Detail
Global Information
Hardware Information
Interface Information
The Device Methods
Utility Fields
Opening and Closing
Packet Transmission
Controlling Transmission Concurrency
Transmission Timeouts
Scatter/Gather I/O
Packet Reception
The Interrupt Handler
Receive Interrupt Mitigation
Changes in Link State
The Socket Buffers
The Important Fields
Functions Acting on Socket Buffers
MAC Address Resolution
Using ARP with Ethernet
Overriding ARP
Non-Ethernet Headers
Custom ioctl Commands
Statistical Information
Multicast
Kernel Support for Multicasting
A Typical Implementation
A Few Other Details
Media Independent Interface Support
Ethtool Support
Netpoll
Quick Reference
Chapter 18
TTY Drivers
A Small TTY Driver
struct termios
tty_driver Function Pointers
open and close
Flow of Data
Other Buffering Functions
No read Function?
TTY Line Settings
set_termios
tiocmget and tiocmset
ioctls
proc and sysfs Handling of TTY Devices
The tty_driver Structure in Detail
The tty_operations Structure in Detail
The tty_struct Structure in Detail
Quick Reference
|
To go directly to, say, page 467, add #467 to the URL, and press enter.
PAGE xi
Preface
This is, on the surface, a book about writing device drivers for the Linux system.
That is a worthy goal, of course; the flow of new hardware products is not likely to
slow down anytime soon, and somebody is going to have to make all those new gadgets
work with Linux. But this book is also about how the Linux kernel works and
how to adapt its workings to your needs or interests. Linux is an open system; with
this book, we hope, it is more open and accessible to a larger community of developers.
This is the third edition of Linux Device Drivers. The kernel has changed greatly
since this book was first published, and we have tried to evolve the text to match.
This edition covers the 2.6.10 kernel as completely as we are able. We have, this time
around, elected to omit the discussion of backward compatibility with previous kernel
versions. The changes from 2.4 are simply too large, and the 2.4 interface
remains well documented in the (freely available) second edition.
This edition contains quite a bit of new material relevant to the 2.6 kernel. The discussion
of locking and concurrency has been expanded and moved into its own
chapter. The Linux device model, which is new in 2.6, is covered in detail. There are
new chapters on the USB bus and the serial driver subsystem; the chapter on PCI has
also been enhanced. While the organization of the rest of the book resembles that of
the earlier editions, every chapter has been thoroughly updated.
We hope you enjoy reading this book as much as we have enjoyed writing it.
Jon's Introduction
The publication of this edition coincides with my twelth year of working with Linux
and, shockingly, my twenty-fifth year in the computing field. Computing seemed like
a fast-moving field back in 1980, but things have sped up a lot since then. Keeping
Linux Device Drivers up to date is increasingly a challenge; the Linux kernel hackers
continue to improve their code, and they have little patience for documentation that
fails to keep up.
PAGE xii
Linux continues to succeed in the market and, more importantly, in the hearts and
minds of developers worldwide. The success of Linux is clearly a testament to its
technical quality and to the numerous benefits of free software in general. But the
true key to its success, in my opinion, lies in the fact that it has brought the fun back
to computing. With Linux, anybody can get their hands into the system and play in a
sandbox where contributions from any direction are welcome, but where technical
excellence is valued above all else. Linux not only provides us with a top-quality
operating system; it gives us the opportunity to be part of its future development and
to have fun while we're at it.
In my 25 years in the field, I have had many interesting opportunities, from programming
the first Cray computers (in Fortran, on punch cards) to seeing the minicomputer
and Unix workstation waves, through to the current, microprocessor-dominated
era. Never, though, have I seen the field more full of life, opportunity,
and fun. Never have we had such control over our own tools and their evolution.
Linux, and free software in general, is clearly the driving force behind those changes.
My hope is that this edition helps to bring that fun and opportunity to a new set of
Linux developers. Whether your interests are in the kernel or in user space, I hope
you find this book to be a useful and interesting guide to just how the kernel works
with the hardware. I hope it helps and inspires you to fire up your editor and to
make our shared, free operating system even better. Linux has come a long way, but
it is also just beginning; it will be more than interesting to watch--and participate
in--what happens from here.
Alessandro's Introduction
I've always enjoyed computers because they can talk to external hardware. So, after
soldering my devices for the Apple II and the ZX Spectrum, backed with the Unix
and free software expertise the university gave me, I could escape the DOS trap by
installing GNU/Linux on a fresh new 386 and by turning on the soldering iron once
again.
Back then, the community was a small one, and there wasn't much documentation
about writing drivers around, so I started writing for Linux Journal. That's how
things started: when I later discovered I didn't like writing papers, I left the univeristy
and found myself with an O'Reilly contract in my hands.
That was in 1996. Ages ago.
The computing world is different now: free software looks like a viable solution,
both technically and politically, but there's a lot of work to do in both realms. I hope
this book furthers two aims: spreading technical knowledge and raising awareness
about the need to spread knowledge. That's why, after the first edition proved interesting
to the public, the two authors of the second edition switched to a free license,
PAGE xiii
supported by our editor and our publisher. I'm betting this is the right approach to
information, and it's great to team up with other people sharing this vision.
I'm excited by what I witness in the embedded arena, and I hope this text helps by
doing more; but ideas are moving fast these days, and it's already time to plan for the
fourth edition, and look for a fourth author to help.
Greg's Introduction
It seems like a long time ago that I picked up the first edition of this Linux Device
Drivers book in order to figure out how to write a real Linux driver. That first edition
was a great guide to helping me understand the internals of this operating system
that I had already been using for a number of years but whose kernel had never
taken the time to look into. With the knowledge gained from that book, and by reading
other programmers' code already present in the kernel, my first horribly buggy,
broken, and very SMP-unsafe driver was accepted by the kernel community into the
main kernel tree. Despite receiving my first bug report five minutes later, I was
hooked on wanting to do as much as I could to make this operating system the best
it could possibly be.
I am honored that I've had the ability to contribute to this book. I hope that it
enables others to learn the details about the kernel, discover that driver development
is not a scary or forbidding place, and possibly encourage others to join in and help
in the collective effort of making this operating system work on every computing
platform with every type of device available. The development procedure is fun, the
community is rewarding, and everyone benefits from the effort involved.
Now it's back to making this edition obsolete by fixing current bugs, changing APIs
to work better and be simpler to understand for everyone, and adding new features.
Come along; we can always use the help.
Audience for This Book
This book should be an interesting source of information both for people who want
to experiment with their computer and for technical programmers who face the need
to deal with the inner levels of a Linux box. Note that "a Linux box" is a wider concept
than "a PC running Linux," as many platforms are supported by our operating
system, and kernel programming is by no means bound to a specific platform. We
hope this book is useful as a starting point for people who want to become kernel
hackers but don't know where to start.
On the technical side, this text should offer a hands-on approach to understanding
the kernel internals and some of the design choices made by the Linux developers.
Although the main, official target of the book is teaching how to write device drivers,
the material should give an interesting overview of the kernel implementation as well.
PAGE xiv
Although real hackers can find all the necessary information in the official kernel
sources, usually a written text can be helpful in developing programming skills. The
text you are approaching is the result of hours of patient grepping through the kernel
sources, and we hope the final result is worth the effort it took.
The Linux enthusiast should find in this book enough food for her mind to start
playing with the code base and should be able to join the group of developers that is
continuously working on new capabilities and performance enhancements. This
book does not cover the Linux kernel in its entirety, of course, but Linux device
driver authors need to know how to work with many of the kernel's subsystems.
Therefore, it makes a good introduction to kernel programming in general. Linux is
still a work in progress, and there's always a place for new programmers to jump into
the game.
If, on the other hand, you are just trying to write a device driver for your own device,
and you don't want to muck with the kernel internals, the text should be modularized
enough to fit your needs as well. If you don't want to go deep into the details,
you can just skip the most technical sections, and stick to the standard API used by
device drivers to seamlessly integrate with the rest of the kernel.
Organization of the Material
The book introduces its topics in ascending order of complexity and is divided into
two parts. The first part (Chapters 1-11) begins with the proper setup of kernel modules
and goes on to describe the various aspects of programming that you'll need in
order to write a full-featured driver for a char-oriented device. Every chapter covers a
distinct problem and includes a quick summary at the end, which can be used as a
reference during actual development.
Throughout the first part of the book, the organization of the material moves roughly
from the software-oriented concepts to the hardware-related ones. This organization
is meant to allow you to test the software on your own computer as far as possible
without the need to plug external hardware into the machine. Every chapter includes
source code and points to sample drivers that you can run on any Linux computer.
In Chapters 9 and 10, however, we ask you to connect an inch of wire to the parallel
port in order to test out hardware handling, but this requirement should be manageable
by everyone.
The second half of the book (Chapters 12-18) describes block drivers and network
interfaces and goes deeper into more advanced topics, such as working with the virtual
memory subsystem and with the PCI and USB buses. Many driver authors do
not need all of this material, but we encourage you to go on reading anyway. Much
of the material found there is interesting as a view into how the Linux kernel works,
even if you do not need it for a specific project.
PAGE xv
Background Information
In order to be able to use this book, you need to be confident with C programming.
Some Unix expertise is needed as well, as we often refer to Unix semantics about system
calls, commands, and pipelines.
At the hardware level, no previous expertise is required to understand the material in
this book, as long as the general concepts are clear in advance. The text isn't based
on specific PC hardware, and we provide all the needed information when we do
refer to specific hardware.
Several free software tools are needed to build the kernel, and you often need specific
versions of these tools. Those that are too old can lack needed features, while
those that are too new can occasionally generate broken kernels. Usually, the tools
provided with any current distribution work just fine. Tool version requirements
vary from one kernel to the next; consult Documentation/Changes in the source tree
of the kernel you are using for exact requirements.
Online Version and License
http://www.oreilly.com/catalog/linuxdrive3
The authors have chosen to make this book freely available under the Creative Commons
"Attribution-ShareAlike" license, Version 2.0:
Attribution-ShareAlike 2.0
You are free:
- to copy, distribute, display, and perform the work
- to make derivative works
- to make commercial use of the work
Under the following conditions:
Attribution. You must attribute the work in the manner specified by the author or licensor.
Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one.
- For any reuse or distribution, you must make clear to others the license terms of this work.
- Any of these conditions can be waived if you get permission from the copyright holder.
Your fair use and other rights are in no way affected by the above.
This is a human-readable summary of the Legal Code (
the full license).
Conventions Used in This Book
The following is a list of the typographical conventions used in this book:
Italic
Used for file and directory names, program and command names, command-line
options, URLs, and new terms
Constant Width
Used in examples to show the contents of files or the output from commands,
and in the text to indicate words that appear in C code or other literal strings
Constant Width Italic
Used to indicate text within commands that the user replaces with an actual
value
Constant Width Bold
Used in examples to show commands or other text that should be typed literally by the user
PAGE xvi
Pay special attention to notes set apart from the text as following:
This is a tip. It contains useful supplementary information about the topic at hand.
This is a warning. It helps you solve and avoid annoying problems.
Using Code Examples
This book is here to help you get your job done. In general, you may use the code in
this book in your programs and documentation. The code samples are covered by a dual BSD/GPL license.
We appreciate, but do not require, attribution. An attribution usually includes the
title, author, publisher, and ISBN. For example: "Linux Device Drivers, Third Edition,
by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman. Copyright
2005 O'Reilly Media, Inc., 0-596-00590-3."
We'd Like to Hear from You
Please address comments and questions concerning this book to the publisher:
O'Reilly Media, Inc.
1005 Gravenstein Highway
North Sebastopol, CA 95472
(800) 998-9938 (in the United States or Canada)
(707) 829-0515 (international or local)
(707) 829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at:
http://www.oreilly.com/catalog/linuxdrive3
To comment or ask technical questions about this book, send email to:
bookquestions@oreilly.com
For more information about our books, conferences, Resource Centers, and the
O'Reilly Network, see our web site at:
http://www.oreilly.com
Safari Enabled
When you see a SafariŽ Enabled icon on the cover of your favorite technology
book, that means the book is available online through the
O'Reilly Network Safari Bookshelf.
Safari offers a solution that's better than e-books. It's a virtual library that lets you
easily search thousands of top tech books, cut and paste code samples, download
chapters, and find quick answers when you need the most accurate, current information.
Try it for free at http://safari.oreilly.com.
Acknowledgments
This book, of course, was not written in a vacuum; we would like to thank the many
people who have helped to make it possible.
Thanks to our editor, Andy Oram; this book is a vastly better product as a result of
his efforts. And obviously we owe a lot to the smart people who have laid the philosophical
and practical foundations of the current free software renaissance.
The first edition was technically reviewed by Alan Cox, Greg Hankins, Hans Lermen,
Heiko Eissfeldt, and Miguel de Icaza (in alphabetic order by first name). The
technical reviewers for the second edition were Allan B. Cruse, Christian Morgner,
Jake Edge, Jeff Garzik, Jens Axboe, Jerry Cooperstein, Jerome Peter Lynch, Michael
Kerrisk, Paul Kinzelman, and Raph Levien. Reviewers for the third edition were
Allan B. Cruse, Christian Morgner, James Bottomley, Jerry Cooperstein, Patrick
Mochel, Paul Kinzelman, and Robert Love. Together, these people have put a vast
amount of effort into finding problems and pointing out possible improvements to
our writing.
Last but certainly not least, we thank the Linux developers for their relentless work.
This includes both the kernel programmers and the user-space people, who often get
forgotten. In this book, we chose never to call them by name in order to avoid being
unfair to someone we might forget. We sometimes made an exception to this rule
and called Linus by name; we hope he doesn't mind.
Jon
I must begin by thanking my wife Laura and my children Michele and Giulia for filling
my life with joy and patiently putting up with my distraction while working on
this edition. The subscribers of LWN.net have, through their generosity, enabled
much of this work to happen. The Linux kernel developers have done me a great service
by letting me be a part of their community, answering my questions, and setting
me straight when I got confused. Thanks are due to readers of the second edition of
this book whose comments, offered at Linux gatherings over much of the world,
PAGE xviii
have been gratifying and inspiring. And I would especially like to thank Alessandro
Rubini for starting this whole exercise with the first edition (and staying with it
through the current edition); and Greg Kroah-Hartman, who has brought his considerable
skills to bear on several chapters, with great results.
Alessandro
I would like to thank the people that made this work possible. First of all, the incredible
patience of Federica, who went as far as letting me review the first edition during
our honeymoon, with a laptop in the tent. I want to thank Giorgio and Giulia,
who have been involved in later editions of the book and happily accepted to be sons
of "a gnu" who often works late in the night. I owe a lot to all the free-software
authors who actually taught me how to program by making their work available for
anyone to study. But for this edition, I'm mostly grateful to Jon and Greg, who have
been great mates in this work; it couldn't have existed without each and both of
them, as the code base is bigger and tougher, while my time is a scarcer resource,
always contended for by clients, free software issues, and expired deadlines. Jon has
been a great leader for this edition; both have been very productive and technically
invaluable in supplementing my small-scale and embedded view toward programming
with their expertise about SMP and number crunchers.
Greg
I would like to thank my wife Shannon and my children Madeline and Griffin for
their understanding and patience while I took the time to work on this book. If it
were not for their support of my original Linux development efforts, I would not be
able to do this book at all. Thanks also to Alessandro and Jon for offering to let me
work on this book; I am honored that they let me participate in it. Much gratitude is
given to all of the Linux kernel programmers, who were unselfish enough to write
code in the public view, so that I and others could learn so much from just reading it.
Also, for everyone who has ever sent me bug reports, critiqued my code, and flamed
me for doing stupid things, you have all taught me so much about how to be a better
programmer and, throughout it all, made me feel very welcome to be part of this
community. Thank you.
PAGE 1
CHAPTER 1
An Introduction to Device Drivers
One of the many advantages of free operating systems, as typified by Linux, is that
their internals are open for all to view. The operating system, once a dark and mysterious
area whose code was restricted to a small number of programmers, can now be
readily examined, understood, and modified by anybody with the requisite skills.
Linux has helped to democratize operating systems. The Linux kernel remains a
large and complex body of code, however, and would-be kernel hackers need an
entry point where they can approach the code without being overwhelmed by complexity.
Often, device drivers provide that gateway.
Device drivers take on a special role in the Linux kernel. They are distinct "black
boxes" that make a particular piece of hardware respond to a well-defined internal
programming interface; they hide completely the details of how the device works.
User activities are performed by means of a set of standardized calls that are independent
of the specific driver; mapping those calls to device-specific operations that act
on real hardware is then the role of the device driver. This programming interface is
such that drivers can be built separately from the rest of the kernel and "plugged in"
at runtime when needed. This modularity makes Linux drivers easy to write, to the
point that there are now hundreds of them available.
There are a number of reasons to be interested in the writing of Linux device drivers.
The rate at which new hardware becomes available (and obsolete!) alone guarantees
that driver writers will be busy for the foreseeable future. Individuals may need to
know about drivers in order to gain access to a particular device that is of interest to
them. Hardware vendors, by making a Linux driver available for their products, can
add the large and growing Linux user base to their potential markets. And the open
source nature of the Linux system means that if the driver writer wishes, the source
to a driver can be quickly disseminated to millions of users.
This book teaches you how to write your own drivers and how to hack around in
related parts of the kernel. We have taken a device-independent approach; the programming
techniques and interfaces are presented, whenever possible, without being
tied to any specific device. Each driver is different; as a driver writer, you need to
PAGE 2
understand your specific device well. But most of the principles and basic techniques
are the same for all drivers. This book cannot teach you about your device,
but it gives you a handle on the background you need to make your device work.
As you learn to write drivers, you find out a lot about the Linux kernel in general;
this may help you understand how your machine works and why things aren't
always as fast as you expect or don't do quite what you want. We introduce new
ideas gradually, starting off with very simple drivers and building on them; every new
concept is accompanied by sample code that doesn't need special hardware to be
tested.
This chapter doesn't actually get into writing code. However, we introduce some
background concepts about the Linux kernel that you'll be glad you know later,
when we do launch into programming.
The Role of the Device Driver
As a programmer, you are able to make your own choices about your driver, and
choose an acceptable trade-off between the programming time required and the flexibility
of the result. Though it may appear strange to say that a driver is "flexible," we
like this word because it emphasizes that the role of a device driver is providing
mechanism, not policy.
The distinction between mechanism and policy is one of the best ideas behind the
Unix design. Most programming problems can indeed be split into two parts: "what
capabilities are to be provided" (the mechanism) and "how those capabilities can be
used" (the policy). If the two issues are addressed by different parts of the program,
or even by different programs altogether, the software package is much easier to
develop and to adapt to particular needs.
For example, Unix management of the graphic display is split between the X server,
which knows the hardware and offers a unified interface to user programs, and the
window and session managers, which implement a particular policy without knowing
anything about the hardware. People can use the same window manager on different
hardware, and different users can run different configurations on the same
workstation. Even completely different desktop environments, such as KDE and
GNOME, can coexist on the same system. Another example is the layered structure
of TCP/IP networking: the operating system offers the socket abstraction, which
implements no policy regarding the data to be transferred, while different servers are
in charge of the services (and their associated policies). Moreover, a server like ftpd
provides the file transfer mechanism, while users can use whatever client they prefer;
both command-line and graphic clients exist, and anyone can write a new user interface
to transfer files.
Where drivers are concerned, the same separation of mechanism and policy applies.
The floppy driver is policy free--its role is only to show the diskette as a continuous
PAGE 3
array of data blocks. Higher levels of the system provide policies, such as who may
access the floppy drive, whether the drive is accessed directly or via a filesystem, and
whether users may mount filesystems on the drive. Since different environments usually
need to use hardware in different ways, it's important to be as policy free as
possible.
When writing drivers, a programmer should pay particular attention to this fundamental
concept: write kernel code to access the hardware, but don't force particular
policies on the user, since different users have different needs. The driver should deal
with making the hardware available, leaving all the issues about how to use the hardware
to the applications. A driver, then, is flexible if it offers access to the hardware
capabilities without adding constraints. Sometimes, however, some policy decisions
must be made. For example, a digital I/O driver may only offer byte-wide access to
the hardware in order to avoid the extra code needed to handle individual bits.
You can also look at your driver from a different perspective: it is a software layer
that lies between the applications and the actual device. This privileged role of the
driver allows the driver programmer to choose exactly how the device should appear:
different drivers can offer different capabilities, even for the same device. The actual
driver design should be a balance between many different considerations. For
instance, a single device may be used concurrently by different programs, and the
driver programmer has complete freedom to determine how to handle concurrency.
You could implement memory mapping on the device independently of its hardware
capabilities, or you could provide a user library to help application programmers
implement new policies on top of the available primitives, and so forth. One major
consideration is the trade-off between the desire to present the user with as many
options as possible and the time you have to write the driver, as well as the need to
keep things simple so that errors don't creep in.
Policy-free drivers have a number of typical characteristics. These include support for
both synchronous and asynchronous operation, the ability to be opened multiple
times, the ability to exploit the full capabilities of the hardware, and the lack of software
layers to "simplify things" or provide policy-related operations. Drivers of this
sort not only work better for their end users, but also turn out to be easier to write
and maintain as well. Being policy-free is actually a common target for software
designers.
Many device drivers, indeed, are released together with user programs to help with
configuration and access to the target device. Those programs can range from simple
utilities to complete graphical applications. Examples include the tunelp program,
which adjusts how the parallel port printer driver operates, and the graphical cardctl
utility that is part of the PCMCIA driver package. Often a client library is provided as
well, which provides capabilities that do not need to be implemented as part of the
driver itself.
PAGE 4
The scope of this book is the kernel, so we try not to deal with policy issues or with
application programs or support libraries. Sometimes we talk about different policies
and how to support them, but we won't go into much detail about programs
using the device or the policies they enforce. You should understand, however, that
user programs are an integral part of a software package and that even policy-free
packages are distributed with configuration files that apply a default behavior to the
underlying mechanisms.
Splitting the Kernel
In a Unix system, several concurrent processes attend to different tasks. Each process
asks for system resources, be it computing power, memory, network connectivity, or
some other resource. The kernel is the big chunk of executable code in charge of handling
all such requests. Although the distinction between the different kernel tasks
isn't always clearly marked, the kernel's role can be split (as shown in Figure 1-1)
into the following parts:
Process management
The kernel is in charge of creating and destroying processes and handling their
connection to the outside world (input and output). Communication among different
processes (through signals, pipes, or interprocess communication primitives)
is basic to the overall system functionality and is also handled by the
kernel. In addition, the scheduler, which controls how processes share the CPU,
is part of process management. More generally, the kernel's process management
activity implements the abstraction of several processes on top of a single
CPU or a few of them.
Memory management
The computer's memory is a major resource, and the policy used to deal with it
is a critical one for system performance. The kernel builds up a virtual addressing
space for any and all processes on top of the limited available resources. The
different parts of the kernel interact with the memory-management subsystem
through a set of function calls, ranging from the simple malloc/free pair to much
more complex functionalities.
Filesystems
Unix is heavily based on the filesystem concept; almost everything in Unix can
be treated as a file. The kernel builds a structured filesystem on top of unstructured
hardware, and the resulting file abstraction is heavily used throughout the
whole system. In addition, Linux supports multiple filesystem types, that is, different
ways of organizing data on the physical medium. For example, disks may
be formatted with the Linux-standard ext3 filesystem, the commonly used FAT
filesystem or several others.
PAGE 5
Device control
Almost every system operation eventually maps to a physical device. With the
exception of the processor, memory, and a very few other entities, any and all
device control operations are performed by code that is specific to the device
being addressed. That code is called a device driver. The kernel must have
embedded in it a device driver for every peripheral present on a system, from the
hard drive to the keyboard and the tape drive. This aspect of the kernel's functions
is our primary interest in this book.
Networking
Networking must be managed by the operating system, because most network
operations are not specific to a process: incoming packets are asynchronous
events. The packets must be collected, identified, and dispatched before a process
takes care of them. The system is in charge of delivering data packets across
program and network interfaces, and it must control the execution of programs
according to their network activity. Additionally, all the routing and address resolution
issues are implemented within the kernel.
Loadable Modules
One of the good features of Linux is the ability to extend at runtime the set of features
offered by the kernel. This means that you can add functionality to the kernel
(and remove functionality as well) while the system is up and running.
Each piece of code that can be added to the kernel at runtime is called a module. The
Linux kernel offers support for quite a few different types (or classes) of modules,
including, but not limited to, device drivers. Each module is made up of object code
(not linked into a complete executable) that can be dynamically linked to the running
kernel by the insmod program and can be unlinked by the rmmod program.
Figure 1-1 identifies different classes of modules in charge of specific tasks--a module
is said to belong to a specific class according to the functionality it offers. The
placement of modules in Figure 1-1 covers the most important classes, but is far from
complete because more and more functionality in Linux is being modularized.
Classes of Devices and Modules
The Linux way of looking at devices distinguishes between three fundamental device
types. Each module usually implements one of these types, and thus is classifiable as a
char module, a block module, or a network module. This division of modules into different
types, or classes, is not a rigid one; the programmer can choose to build huge
modules implementing different drivers in a single chunk of code. Good programmers,
nonetheless, usually create a different module for each new functionality they
implement, because decomposition is a key element of scalability and extendability.
PAGE 6
Figure 1-1. A split view of the kernel
The three classes are:
Character devices
A character (char) device is one that can be accessed as a stream of bytes (like a
file); a char driver is in charge of implementing this behavior. Such a driver usually
implements at least the open, close, read, and write system calls. The text
console (/dev/console) and the serial ports (/dev/ttyS0 and friends) are examples
of char devices, as they are well represented by the stream abstraction. Char
devices are accessed by means of filesystem nodes, such as /dev/tty1 and /dev/lp0.
The only relevant difference between a char device and a regular file is that you
can always move back and forth in the regular file, whereas most char devices
are just data channels, which you can only access sequentially. There exist,
nonetheless, char devices that look like data areas, and you can move back and
forth in them; for instance, this usually applies to frame grabbers, where the
applications can access the whole acquired image using mmap or lseek.
PAGE 7
Block devices
Like char devices, block devices are accessed by filesystem nodes in the /dev
directory. A block device is a device (e.g., a disk) that can host a filesystem. In
most Unix systems, a block device can only handle I/O operations that transfer
one or more whole blocks, which are usually 512 bytes (or a larger power of
two) bytes in length. Linux, instead, allows the application to read and write a
block device like a char device--it permits the transfer of any number of bytes at
a time. As a result, block and char devices differ only in the way data is managed
internally by the kernel, and thus in the kernel/driver software interface. Like a
char device, each block device is accessed through a filesystem node, and the difference
between them is transparent to the user. Block drivers have a completely
different interface to the kernel than char drivers.
Network interfaces
Any network transaction is made through an interface, that is, a device that is
able to exchange data with other hosts. Usually, an interface is a hardware
device, but it might also be a pure software device, like the loopback interface. A
network interface is in charge of sending and receiving data packets, driven by
the network subsystem of the kernel, without knowing how individual transactions
map to the actual packets being transmitted. Many network connections
(especially those using TCP) are stream-oriented, but network devices are, usually,
designed around the transmission and receipt of packets. A network driver
knows nothing about individual connections; it only handles packets.
Not being a stream-oriented device, a network interface isn't easily mapped to a
node in the filesystem, as /dev/tty1 is. The Unix way to provide access to interfaces
is still by assigning a unique name to them (such as eth0), but that name
doesn't have a corresponding entry in the filesystem. Communication between
the kernel and a network device driver is completely different from that used
with char and block drivers. Instead of read and write, the kernel calls functions
related to packet transmission.
There are other ways of classifying driver modules that are orthogonal to the above
device types. In general, some types of drivers work with additional layers of kernel
support functions for a given type of device. For example, one can talk of universal
serial bus (USB) modules, serial modules, SCSI modules, and so on. Every USB
device is driven by a USB module that works with the USB subsystem, but the device
itself shows up in the system as a char device (a USB serial port, say), a block device
(a USB memory card reader), or a network device (a USB Ethernet interface).
Other classes of device drivers have been added to the kernel in recent times, including
FireWire drivers and I2C drivers. In the same way that they handled USB and
SCSI drivers, kernel developers collected class-wide features and exported them to
driver implementers to avoid duplicating work and bugs, thus simplifying and
strengthening the process of writing such drivers.
PAGE 8
In addition to device drivers, other functionalities, both hardware and software, are
modularized in the kernel. One common example is filesystems. A filesystem type
determines how information is organized on a block device in order to represent a
tree of directories and files. Such an entity is not a device driver, in that there's no
explicit device associated with the way the information is laid down; the filesystem
type is instead a software driver, because it maps the low-level data structures to
high-level data structures. It is the filesystem that determines how long a filename
can be and what information about each file is stored in a directory entry. The filesystem
module must implement the lowest level of the system calls that access directories
and files, by mapping filenames and paths (as well as other information, such
as access modes) to data structures stored in data blocks. Such an interface is completely
independent of the actual data transfer to and from the disk (or other
medium), which is accomplished by a block device driver.
If you think of how strongly a Unix system depends on the underlying filesystem,
you'll realize that such a software concept is vital to system operation. The ability to
decode filesystem information stays at the lowest level of the kernel hierarchy and is
of utmost importance; even if you write a block driver for your new CD-ROM, it is
useless if you are not able to run ls or cp on the data it hosts. Linux supports the concept
of a filesystem module, whose software interface declares the different operations
that can be performed on a filesystem inode, directory, file, and superblock. It's
quite unusual for a programmer to actually need to write a filesystem module,
because the official kernel already includes code for the most important filesystem
types.
Security Issues
Security is an increasingly important concern in modern times. We will discuss security-related
issues as they come up throughout the book. There are a few general concepts,
however, that are worth mentioning now.
Any security check in the system is enforced by kernel code. If the kernel has security
holes, then the system as a whole has holes. In the official kernel distribution,
only an authorized user can load modules; the system call init_module checks if the
invoking process is authorized to load a module into the kernel. Thus, when running
an official kernel, only the superuser,* or an intruder who has succeeded in
becoming privileged, can exploit the power of privileged code.
When possible, driver writers should avoid encoding security policy in their code.
Security is a policy issue that is often best handled at higher levels within the kernel,
under the control of the system administrator. There are always exceptions, however.
* Technically, only somebody with the CAP_SYS_MODULE capability
can perform this operation. We discuss capabilities in Chapter 6.
PAGE 9
As a device driver writer, you should be aware of situations in which some types of
device access could adversely affect the system as a whole and should provide adequate
controls. For example, device operations that affect global resources (such as
setting an interrupt line), which could damage the hardware (loading firmware, for
example), or that could affect other users (such as setting a default block size on a
tape drive), are usually only available to sufficiently privileged users, and this check
must be made in the driver itself.
Driver writers must also be careful, of course, to avoid introducing security bugs.
The C programming language makes it easy to make several types of errors. Many
current security problems are created, for example, by buffer overrun errors, in which
the programmer forgets to check how much data is written to a buffer, and data ends
up written beyond the end of the buffer, thus overwriting unrelated data. Such errors
can compromise the entire system and must be avoided. Fortunately, avoiding these
errors is usually relatively easy in the device driver context, in which the interface to
the user is narrowly defined and highly controlled.
Some other general security ideas are worth keeping in mind. Any input received
from user processes should be treated with great suspicion; never trust it unless you
can verify it. Be careful with uninitialized memory; any memory obtained from the
kernel should be zeroed or otherwise initialized before being made available to a user
process or device. Otherwise, information leakage (disclosure of data, passwords,
etc.) could result. If your device interprets data sent to it, be sure the user cannot
send anything that could compromise the system. Finally, think about the possible
effect of device operations; if there are specific operations (e.g., reloading the firmware
on an adapter board or formatting a disk) that could affect the system, those
operations should almost certainly be restricted to privileged users.
Be careful, also, when receiving software from third parties, especially when the kernel
is concerned: because everybody has access to the source code, everybody can
break and recompile things. Although you can usually trust precompiled kernels
found in your distribution, you should avoid running kernels compiled by an
untrusted friend--if you wouldn't run a precompiled binary as root, then you'd better
not run a precompiled kernel. For example, a maliciously modified kernel could
allow anyone to load a module, thus opening an unexpected back door via init_module.
Note that the Linux kernel can be compiled to have no module support whatsoever,
thus closing any module-related security holes. In this case, of course, all needed
drivers must be built directly into the kernel itself. It is also possible, with 2.2 and
later kernels, to disable the loading of kernel modules after system boot via the capability
mechanism.
PAGE 10
Version Numbering
Before digging into programming, we should comment on the version numbering
scheme used in Linux and which versions are covered by this book.
First of all, note that every software package used in a Linux system has its own
release number, and there are often interdependencies across them: you need a particular
version of one package to run a particular version of another package. The
creators of Linux distributions usually handle the messy problem of matching packages,
and the user who installs from a prepackaged distribution doesn't need to deal
with version numbers. Those who replace and upgrade system software, on the other
hand, are on their own in this regard. Fortunately, almost all modern distributions
support the upgrade of single packages by checking interpackage dependencies; the
distribution's package manager generally does not allow an upgrade until the dependencies
are satisfied.
To run the examples we introduce during the discussion, you won't need particular
versions of any tool beyond what the 2.6 kernel requires; any recent Linux distribution
can be used to run our examples. We won't detail specific requirements,
because the file Documentation/Changes in your kernel sources is the best source of
such information if you experience any problems.
As far as the kernel is concerned, the even-numbered kernel versions (i.e., 2.6.x) are
the stable ones that are intended for general distribution. The odd versions (such as
2.7.x), on the contrary, are development snapshots and are quite ephemeral; the latest
of them represents the current status of development, but becomes obsolete in a
few days or so.
This book covers Version 2.6 of the kernel. Our focus has been to show all the features
available to device driver writers in 2.6.10, the current version at the time we
are writing. This edition of the book does not cover prior versions of the kernel. For
those of you who are interested, the second edition covered Versions 2.0 through 2.4
in detail. That edition is still available online at http://lwn.net/Kernel/LDD2/.
Kernel programmers should be aware that the development process changed with 2.6.
The 2.6 series is now accepting changes that previously would have been considered
too large for a "stable" kernel. Among other things, that means that internal kernel
programming interfaces can change, thus potentially obsoleting parts of this book;
for this reason, the sample code accompanying the text is known to work with 2.6.10,
but some modules don't compile under earlier versions. Programmers wanting to
keep up with kernel programming changes are encouraged to join the mailing lists
and to make use of the web sites listed in the bibliography. There is also a web page
maintained at http://lwn.net/Articles/2.6-kernel-api/, which contains information
about API changes that have happened since this book was published.
PAGE 11
This text doesn't talk specifically about odd-numbered kernel versions. General users
never have a reason to run development kernels. Developers experimenting with new
features, however, want to be running the latest development release. They usually
keep upgrading to the most recent version to pick up bug fixes and new implementations
of features. Note, however, that there's no guarantee on experimental kernels,*
and nobody helps you if you have problems due to a bug in a noncurrent odd-numbered
kernel. Those who run odd-numbered versions of the kernel are usually skilled
enough to dig in the code without the need for a textbook, which is another reason
why we don't talk about development kernels here.
Another feature of Linux is that it is a platform-independent operating system, not
just "a Unix clone for PC clones" anymore: it currently supports some 20 architectures.
This book is platform independent as far as possible, and all the code samples
have been tested on at least the x86 and x86-64 platforms. Because the code has been
tested on both 32-bit and 64-bit processors, it should compile and run on all other
platforms. As you might expect, the code samples that rely on particular hardware
don't work on all the supported platforms, but this is always stated in the source
code.
License Terms
Linux is licensed under Version 2 of the GNU General Public License (GPL), a document
devised for the GNU project by the Free Software Foundation. The GPL allows
anybody to redistribute, and even sell, a product covered by the GPL, as long as the
recipient has access to the source and is able to exercise the same rights. Additionally,
any software product derived from a product covered by the GPL must, if it is
redistributed at all, be released under the GPL.
The main goal of such a license is to allow the growth of knowledge by permitting
everybody to modify programs at will; at the same time, people selling software to
the public can still do their job. Despite this simple objective, there's a never-ending
discussion about the GPL and its use. If you want to read the license, you can find it
in several places in your system, including the top directory of your kernel source
tree in the COPYING file.
Vendors often ask whether they can distribute kernel modules in binary form only.
The answer to that question has been deliberately left ambiguous. Distribution of
binary modules--as long as they adhere to the published kernel interface--has been
tolerated so far. But the copyrights on the kernel are held by many developers, and
not all of them agree that kernel modules are not derived products. If you or your
employer wish to distribute kernel modules under a nonfree license, you really need
* Note that there's no guarantee on even-numbered kernels as well, unless you rely
on a commercial provider that grants its own warranty.
PAGE 12
to discuss the situation with your legal counsel. Please note also that the kernel
developers have no qualms against breaking binary modules between kernel releases,
even in the middle of a stable kernel series. If it is at all possible, both you and your
users are better off if you release your module as free software.
If you want your code to go into the mainline kernel, or if your code requires patches
to the kernel, you must use a GPL-compatible license as soon as you release the code.
Although personal use of your changes doesn't force the GPL on you, if you distribute
your code, you must include the source code in the distribution--people acquiring
your package must be allowed to rebuild the binary at will.
As far as this book is concerned, most of the code is freely redistributable, either in
source or binary form, and neither we nor O'Reilly retain any right on any derived
works. All the programs are available at
ftp://ftp.ora.com/pub/examples/linux/drivers/,
and the exact license terms are stated in the LICENSE file in the same directory.
Joining the Kernel Development Community
As you begin writing modules for the Linux kernel, you become part of a larger community
of developers. Within that community, you can find not only people engaged
in similar work, but also a group of highly committed engineers working toward
making Linux a better system. These people can be a source of help, ideas, and critical
review as well--they will be the first people you will likely turn to when you are
looking for testers for a new driver.
The central gathering point for Linux kernel developers is the linux-kernel mailing
list. All major kernel developers, from Linus Torvalds on down, subscribe to this list.
Please note that the list is not for the faint of heart: traffic as of this writing can run
up to 200 messages per day or more. Nonetheless, following this list is essential for
those who are interested in kernel development; it also can be a top-quality resource
for those in need of kernel development help.
To join the linux-kernel list, follow the instructions found in the linux-kernel mailing
list FAQ:
http://www.tux.org/lkml.
Read the rest of the FAQ while you are at it;
there is a great deal of useful information there. Linux kernel developers are busy
people, and they are much more inclined to help people who have clearly done their
homework first.
Overview of the Book
From here on, we enter the world of kernel programming. Chapter 2 introduces
modularization, explaining the secrets of the art and showing the code for running
modules. Chapter 3 talks about char drivers and shows the complete code for a
PAGE 13
memory-based device driver that can be read and written for fun. Using memory as
the hardware base for the device allows anyone to run the sample code without the
need to acquire special hardware.
Debugging techniques are vital tools for the programmer and are introduced in
Chapter 4. Equally important for those who would hack on contemporary kernels is
the management of concurrency and race conditions. Chapter 5 concerns itself with
the problems posed by concurrent access to resources and introduces the Linux
mechanisms for controlling concurrency.
With debugging and concurrency management skills in place, we move to advanced
features of char drivers, such as blocking operations, the use of select, and the important
ioctl call; these topics are the subject of Chapter 6.
Before dealing with hardware management, we dissect a few more of the kernel's
software interfaces: Chapter 7 shows how time is managed in the kernel, and
Chapter 8 explains memory allocation.
Next we focus on hardware. Chapter 9 describes the management of I/O ports and
memory buffers that live on the device; after that comes interrupt handling, in
Chapter 10. Unfortunately, not everyone is able to run the sample code for these
chapters, because some hardware support is actually needed to test the software
interface interrupts. We've tried our best to keep required hardware support to a
minimum, but you still need some simple hardware, such as a standard parallel port,
to work with the sample code for these chapters.
Chapter 11 covers the use of data types in the kernel and the writing of portable
code.
The second half of the book is dedicated to more advanced topics. We start by getting
deeper into the hardware and, in particular, the functioning of specific peripheral
buses. Chapter 12 covers the details of writing drivers for PCI devices, and
Chapter 13 examines the API for working with USB devices.
With an understanding of peripheral buses in place, we can take a detailed look at the
Linux device model, which is the abstraction layer used by the kernel to describe the
hardware and software resources it is managing. Chapter 14 is a bottom-up look at
the device model infrastructure, starting with the kobject type and working up from
there. It covers the integration of the device model with real hardware; it then uses
that knowledge to cover topics like hot-pluggable devices and power management.
In Chapter 15, we take a diversion into Linux memory management. This chapter
shows how to map kernel memory into user space (the mmap system call), map user
memory into kernel space (with get_user_pages), and how to map either kind of
memory into device space (to perform direct memory access [DMA] operations).
PAGE 14
Our understanding of memory will be useful for the following two chapters, which
cover the other major driver classes. Chapter 16 introduces block drivers and shows
how they are different from the char drivers we have worked with so far. Then
Chapter 17 gets into the writing of network drivers. We finish up with a discussion
of serial drivers (Chapter 18) and a bibliography.
PAGE 15
CHAPTER 2
Building and Running Modules
It's almost time to begin programming. This chapter introduces all the essential concepts
about modules and kernel programming. In these few pages, we build and run
a complete (if relatively useless) module, and look at some of the basic code shared
by all modules. Developing such expertise is an essential foundation for any kind of
modularized driver. To avoid throwing in too many concepts at once, this chapter
talks only about modules, without referring to any specific device class.
All the kernel items (functions, variables, header files, and macros) that are introduced
here are described in a reference section at the end of the chapter.
Setting Up Your Test System
Starting with this chapter, we present example modules to demonstrate programming
concepts. (All of these examples are available on O'Reilly's FTP site, as
explained in Chapter 1.) Building, loading, and modifying these examples are a good
way to improve your understanding of how drivers work and interact with the kernel.
The example modules should work with almost any 2.6.x kernel, including those
provided by distribution vendors. However, we recommend that you obtain a "mainline"
kernel directly from the kernel.org mirror network, and install it on your system.
Vendor kernels can be heavily patched and divergent from the mainline; at
times, vendor patches can change the kernel API as seen by device drivers. If you are
writing a driver that must work on a particular distribution, you will certainly want
to build and test against the relevant kernels. But, for the purpose of learning about
driver writing, a standard kernel is best.
Regardless of the origin of your kernel, building modules for 2.6.x requires that you
have a configured and built kernel tree on your system. This requirement is a change
from previous versions of the kernel, where a current set of header files was sufficient.
2.6 modules are linked against object files found in the kernel source tree; the
result is a more robust module loader, but also the requirement that those object files
PAGE 16
be available. So your first order of business is to come up with a kernel source tree
(either from the kernel.org network or your distributor's kernel source package),
build a new kernel, and install it on your system. For reasons we'll see later, life is
generally easiest if you are actually running the target kernel when you build your
modules, though this is not required.
You should also give some thought to where you do your module
experimentation, development, and testing. We have done our best to
make our example modules safe and correct, but the possibility of
bugs is always present. Faults in kernel code can bring about the
demise of a user process or, occasionally, the entire system. They do
not normally create more serious problems, such as disk corruption.
Nonetheless, it is advisable to do your kernel experimentation on a
system that does not contain data that you cannot afford to lose, and
that does not perform essential services. Kernel hackers typically keep
a "sacrificial" system around for the purpose of testing new code.
So, if you do not yet have a suitable system with a configured and built kernel source
tree on disk, now would be a good time to set that up. We'll wait. Once that task is
taken care of, you'll be ready to start playing with kernel modules.
The Hello World Module
Many programming books begin with a "hello world" example as a way of showing
the simplest possible program. This book deals in kernel modules rather than programs;
so, for the impatient reader, the following code is a complete "hello world"
module:
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("Dual BSD/GPL");
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
This module defines two functions, one to be invoked when the module is loaded
into the kernel (hello_init) and one for when the module is removed (hello_exit). The
PAGE 17
module_init and module_exit lines use special kernel macros to indicate the role of
these two functions. Another special macro (MODULE_LICENSE) is used to tell the
kernel that this module bears a free license; without such a declaration, the kernel
complains when the module is loaded.
The printk function is defined in the Linux kernel and made available to modules; it
behaves similarly to the standard C library function printf. The kernel needs its own
printing function because it runs by itself, without the help of the C library. The
module can call printk because, after insmod has loaded it, the module is linked to
the kernel and can access the kernel's public symbols (functions and variables, as
detailed in the next section). The string KERN_ALERT is the priority of the message.*
We've specified a high priority in this module, because a message with the default
priority might not show up anywhere useful, depending on the kernel version you
are running, the version of the klogd daemon, and your configuration. You can
ignore this issue for now; we explain it in Chapter 4.
You can test the module with the insmod and rmmod utilities, as shown below. Note
that only the superuser can load and unload a module.
% make
make[1]: Entering directory `/usr/src/linux-2.6.10'
CC [M] /home/ldd3/src/misc-modules/hello.o
Building modules, stage 2.
MODPOST
CC /home/ldd3/src/misc-modules/hello.mod.o
LD [M] /home/ldd3/src/misc-modules/hello.ko
make[1]: Leaving directory `/usr/src/linux-2.6.10'
% su
root# insmod ./hello.ko
Hello, world
root# rmmod hello
Goodbye, cruel world
root#
Please note once again that, for the above sequence of commands to work, you must
have a properly configured and built kernel tree in a place where the makefile is able
to find it (/usr/src/linux-2.6.10 in the example shown). We get into the details of how
modules are built in the section "Compiling and Loading."
According to the mechanism your system uses to deliver the message lines, your output
may be different. In particular, the previous screen dump was taken from a text
console; if you are running insmod and rmmod from a terminal emulator running
under the window system, you won't see anything on your screen. The message goes
to one of the system log files, such as /var/log/messages (the name of the actual file
* The priority is just a string, such as <1>, which is prepended to the printk format
string. Note the lack of a comma after KERN_ALERT; adding a comma there is a common and annoying
typo (which, fortunately, is caught by the compiler).
PAGE 18
varies between Linux distributions). The mechanism used to deliver kernel messages
is described in Chapter 4.
As you can see, writing a module is not as difficult as you might expect--at least, as
long as the module is not required to do anything worthwhile. The hard part is
understanding your device and how to maximize performance. We go deeper into
modularization throughout this chapter and leave device-specific issues for later
chapters.
Kernel Modules Versus Applications
Before we go further, it's worth underlining the various differences between a kernel
module and an application.
While most small and medium-sized applications perform a single task from beginning
to end, every kernel module just registers itself in order to serve future requests,
and its initialization function terminates immediately. In other words, the task of the
module's initialization function is to prepare for later invocation of the module's
functions; it's as though the module were saying, "Here I am, and this is what I can
do." The module's exit function (hello_exit in the example) gets invoked just before
the module is unloaded. It should tell the kernel, "I'm not there anymore; don't ask
me to do anything else." This kind of approach to programming is similar to event-driven
programming, but while not all applications are event-driven, each and every
kernel module is. Another major difference between event-driven applications and
kernel code is in the exit function: whereas an application that terminates can be lazy
in releasing resources or avoids clean up altogether, the exit function of a module
must carefully undo everything the init function built up, or the pieces remain
around until the system is rebooted.
Incidentally, the ability to unload a module is one of the features of modularization
that you'll most appreciate, because it helps cut down development time; you can
test successive versions of your new driver without going through the lengthy shutdown/reboot
cycle each time.
As a programmer, you know that an application can call functions it doesn't define:
the linking stage resolves external references using the appropriate library of functions.
printf is one of those callable functions and is defined in libc. A module, on the
other hand, is linked only to the kernel, and the only functions it can call are the
ones exported by the kernel; there are no libraries to link to. The printk function
used in hello.c earlier, for example, is the version of printf defined within the kernel
and exported to modules. It behaves similarly to the original function, with a few
minor differences, the main one being lack of floating-point support.
Figure 2-1 shows how function calls and function pointers are used in a module to
add new functionality to a running kernel.
PAGE 19
Figure 2-1. Linking a module to the kernel
Because no library is linked to modules, source files should never include the usual
header files, <stdarg.h> and very special situations being the only exceptions. Only
functions that are actually part of the kernel itself may be used in kernel modules.
Anything related to the kernel is declared in headers found in the kernel source tree
you have set up and configured; most of the relevant headers live in include/linux and
include/asm, but other subdirectories of include have been added to host material
associated to specific kernel subsystems.
The role of individual kernel headers is introduced throughout the book as each of
them is needed.
Another important difference between kernel programming and application programming
is in how each environment handles faults: whereas a segmentation fault
is harmless during application development and a debugger can always be used to
trace the error to the problem in the source code, a kernel fault kills the current process
at least, if not the whole system. We see how to trace kernel errors in Chapter 4.
User Space and Kernel Space
A module runs in kernel space, whereas applications run in user space. This concept
is at the base of operating systems theory.
The role of the operating system, in practice, is to provide programs with a consistent
view of the computer's hardware. In addition, the operating system must
account for independent operation of programs and protection against unauthorized
access to resources. This nontrivial task is possible only if the CPU enforces protection
of system software from the applications.
PAGE 20
Every modern processor is able to enforce this behavior. The chosen approach is to
implement different operating modalities (or levels) in the CPU itself. The levels have
different roles, and some operations are disallowed at the lower levels; program code
can switch from one level to another only through a limited number of gates. Unix
systems are designed to take advantage of this hardware feature, using two such levels.
All current processors have at least two protection levels, and some, like the x86
family, have more levels; when several levels exist, the highest and lowest levels are
used. Under Unix, the kernel executes in the highest level (also called supervisor
mode), where everything is allowed, whereas applications execute in the lowest level
(the so-called user mode), where the processor regulates direct access to hardware
and unauthorized access to memory.
We usually refer to the execution modes as kernel space and user space. These terms
encompass not only the different privilege levels inherent in the two modes, but also
the fact that each mode can have its own memory mapping--its own address
space--as well.
Unix transfers execution from user space to kernel space whenever an application
issues a system call or is suspended by a hardware interrupt. Kernel code executing a
system call is working in the context of a process--it operates on behalf of the calling
process and is able to access data in the process's address space. Code that handles
interrupts, on the other hand, is asynchronous with respect to processes and is
not related to any particular process.
The role of a module is to extend kernel functionality; modularized code runs in kernel
space. Usually a driver performs both the tasks outlined previously: some functions
in the module are executed as part of system calls, and some are in charge of
interrupt handling.
Concurrency in the Kernel
One way in which kernel programming differs greatly from conventional application
programming is the issue of concurrency. Most applications, with the notable exception
of multithreading applications, typically run sequentially, from the beginning to
the end, without any need to worry about what else might be happening to change
their environment. Kernel code does not run in such a simple world, and even the
simplest kernel modules must be written with the idea that many things can be happening
at once.
There are a few sources of concurrency in kernel programming. Naturally, Linux systems
run multiple processes, more than one of which can be trying to use your driver
at the same time. Most devices are capable of interrupting the processor; interrupt
handlers run asynchronously and can be invoked at the same time that your driver is
trying to do something else. Several software abstractions (such as kernel timers,
introduced in Chapter 7) run asynchronously as well. Moreover, of course, Linux
PAGE 21
can run on symmetric multiprocessor (SMP) systems, with the result that your driver
could be executing concurrently on more than one CPU. Finally, in 2.6, kernel code
has been made preemptible; this change causes even uniprocessor systems to have
many of the same concurrency issues as multiprocessor systems.
As a result, Linux kernel code, including driver code, must be reentrant--it must be
capable of running in more than one context at the same time. Data structures must
be carefully designed to keep multiple threads of execution separate, and the code
must take care to access shared data in ways that prevent corruption of the data.
Writing code that handles concurrency and avoids race conditions (situations in
which an unfortunate order of execution causes undesirable behavior) requires
thought and can be tricky. Proper management of concurrency is required to write
correct kernel code; for that reason, every sample driver in this book has been written
with concurrency in mind. The techniques used are explained as we come to
them; Chapter 5 has also been dedicated to this issue and the kernel primitives available
for concurrency management.
A common mistake made by driver programmers is to assume that concurrency is
not a problem as long as a particular segment of code does not go to sleep (or
"block"). Even in previous kernels (which were not preemptive), this assumption
was not valid on multiprocessor systems. In 2.6, kernel code can (almost) never
assume that it can hold the processor over a given stretch of code. If you do not write
your code with concurrency in mind, it will be subject to catastrophic failures that
can be exceedingly difficult to debug.
The Current Process
Although kernel modules don't execute sequentially as applications do, most actions
performed by the kernel are done on behalf of a specific process. Kernel code can
refer to the current process by accessing the global item current, defined in <asm/
current.h>, which yields a pointer to struct task_struct, defined by <linux/sched.h>.
The current pointer refers to the process that is currently executing. During the execution
of a system call, such as open or read, the current process is the one that
invoked the call. Kernel code can use process-specific information by using current,
if it needs to do so. An example of this technique is presented in Chapter 6.
Actually, current is not truly a global variable. The need to support SMP systems
forced the kernel developers to develop a mechanism that finds the current process on
the relevant CPU. This mechanism must also be fast, since references to current happen
frequently. The result is an architecture-dependent mechanism that, usually,
hides a pointer to the task_struct structure on the kernel stack. The details of the
implementation remain hidden to other kernel subsystems though, and a device
driver can just include <linux/sched.h> and refer to the current process. For example,
PAGE 22
the following statement prints the process ID and the command name of the current
process by accessing certain fields in struct task_struct:
printk(KERN_INFO "The process is \"%s\" (pid %i)\n",
current->comm, current->pid);
The command name stored in current->comm is the base name of the program file
(trimmed to 15 characters if need be) that is being executed by the current process.
A Few Other Details
Kernel programming differs from user-space programming in many ways. We'll
point things out as we get to them over the course of the book, but there are a few
fundamental issues which, while not warranting a section of their own, are worth a
mention. So, as you dig into the kernel, the following issues should be kept in mind.
Applications are laid out in virtual memory with a very large stack area. The stack, of
course, is used to hold the function call history and all automatic variables created by
currently active functions. The kernel, instead, has a very small stack; it can be as
small as a single, 4096-byte page. Your functions must share that stack with the
entire kernel-space call chain. Thus, it is never a good idea to declare large automatic
variables; if you need larger structures, you should allocate them dynamically
at call time.
Often, as you look at the kernel API, you will encounter function names starting with
a double underscore (__). Functions so marked are generally a low-level component
of the interface and should be used with caution. Essentially, the double underscore
says to the programmer: "If you call this function, be sure you know what you are
doing."
Kernel code cannot do floating point arithmetic. Enabling floating point would
require that the kernel save and restore the floating point processor's state on each
entry to, and exit from, kernel space--at least, on some architectures. Given that
there really is no need for floating point in kernel code, the extra overhead is not
worthwhile.
Compiling and Loading
The "hello world" example at the beginning of this chapter included a brief demonstration
of building a module and loading it into the system. There is, of course, a lot
more to that whole process than we have seen so far. This section provides more
detail on how a module author turns source code into an executing subsystem within
the kernel.
PAGE 23
Compiling Modules
As the first step, we need to look a bit at how modules must be built. The build process
for modules differs significantly from that used for user-space applications; the
kernel is a large, stand alone program with detailed and explicit requirements on how
its pieces are put together. The build process also differs from how things were done
with previous versions of the kernel; the new build system is simpler to use and produces
more correct results, but it looks very different from what came before. The
kernel build system is a complex beast, and we just look at a tiny piece of it. The files
found in the Documentation/kbuild directory in the kernel source are required reading
for anybody wanting to understand all that is really going on beneath the surface.
There are some prerequisites that you must get out of the way before you can build
kernel modules. The first is to ensure that you have sufficiently current versions of the
compiler, module utilities, and other necessary tools. The file Documentation/Changes
in the kernel documentation directory always lists the required tool versions; you
should consult it before going any further. Trying to build a kernel (and its modules)
with the wrong tool versions can lead to no end of subtle, difficult problems. Note
that, occasionally, a version of the compiler that is too new can be just as problematic
as one that is too old; the kernel source makes a great many assumptions about the
compiler, and new releases can sometimes break things for a while.
If you still do not have a kernel tree handy, or have not yet configured and built that
kernel, now is the time to go do it. You cannot build loadable modules for a 2.6 kernel
without this tree on your filesystem. It is also helpful (though not required) to be
actually running the kernel that you are building for.
Once you have everything set up, creating a makefile for your module is straightforward.
In fact, for the "hello world" example shown earlier in this chapter, a single
line will suffice:
obj-m := hello.o
Readers who are familiar with make, but not with the 2.6 kernel build system, are
likely to be wondering how this makefile works. The above line is not how a traditional
makefile looks, after all. The answer, of course, is that the kernel build system
handles the rest. The assignment above (which takes advantage of the extended syntax
provided by GNU make) states that there is one module to be built from the
object file hello.o. The resulting module is named hello.ko after being built from the
object file.
If, instead, you have a module called module.ko that is generated from two source
files (called, say, file1.c and file2.c), the correct incantation would be:
obj-m := module.o
module-objs := file1.o file2.o
For a makefile like those shown above to work, it must be invoked within the context
of the larger kernel build system. If your kernel source tree is located in, say,
PAGE 24
your ~/kernel-2.6 directory, the make command required to build your module
(typed in the directory containing the module source and makefile) would be:
make -C ~/kernel-2.6 M=`pwd` modules
This command starts by changing its directory to the one provided with the -C
option (that is, your kernel source directory). There it finds the kernel's top-level
makefile. The M= option causes that makefile to move back into your module source
directory before trying to build the modules target. This target, in turn, refers to the list
of modules found in the obj-m variable, which we've set to module.o in our examples.
Typing the previous make command can get tiresome after a while, so the kernel
developers have developed a sort of makefile idiom, which makes life easier for those
building modules outside of the kernel tree. The trick is to write your makefile as follows:
# If KERNELRELEASE is defined, we've been invoked from the
# kernel build system and can use its language.
ifneq ($(KERNELRELEASE),)
obj-m := hello.o
# Otherwise we were called directly from the command
# line; invoke the kernel build system.
else
KERNELDIR ?= /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
endif
Once again, we are seeing the extended GNU make syntax in action. This makefile is
read twice on a typical build. When the makefile is invoked from the command line,
it notices that the KERNELRELEASE variable has not been set. It locates the kernel source
directory by taking advantage of the fact that the symbolic link build in the installed
modules directory points back at the kernel build tree. If you are not actually running
the kernel that you are building for, you can supply a KERNELDIR= option on the
command line, set the KERNELDIR environment variable, or rewrite the line that sets
KERNELDIR in the makefile. Once the kernel source tree has been found, the makefile
invokes the default: target, which runs a second make command (parameterized in
the makefile as $(MAKE)) to invoke the kernel build system as described previously.
On the second reading, the makefile sets obj-m, and the kernel makefiles take care of
actually building the module.
This mechanism for building modules may strike you as a bit unwieldy and obscure.
Once you get used to it, however, you will likely appreciate the capabilities that have
been programmed into the kernel build system. Do note that the above is not a complete
makefile; a real makefile includes the usual sort of targets for cleaning up
PAGE 25
unneeded files, installing modules, etc. See the makefiles in the example source
directory for a complete example.
Loading and Unloading Modules
After the module is built, the next step is loading it into the kernel. As we've already
pointed out, insmod does the job for you. The program loads the module code and
data into the kernel, which, in turn, performs a function similar to that of ld, in that
it links any unresolved symbol in the module to the symbol table of the kernel.
Unlike the linker, however, the kernel doesn't modify the module's disk file, but
rather an in-memory copy. insmod accepts a number of command-line options (for
details, see the manpage), and it can assign values to parameters in your module
before linking it to the current kernel. Thus, if a module is correctly designed, it can
be configured at load time; load-time configuration gives the user more flexibility
than compile-time configuration, which is still used sometimes. Load-time configuration
is explained in the section "Module Parameters," later in this chapter.
Interested readers may want to look at how the kernel supports insmod: it relies on a
system call defined in kernel/module.c. The function sys_init_module allocates kernel
memory to hold a module (this memory is allocated with vmalloc; see the section
"vmalloc and Friends" in Chapter 8); it then copies the module text into that memory
region, resolves kernel references in the module via the kernel symbol table, and
calls the module's initialization function to get everything going.
If you actually look in the kernel source, you'll find that the names of the system calls
are prefixed with sys_. This is true for all system calls and no other functions; it's
useful to keep this in mind when grepping for the system calls in the sources.
The modprobe utility is worth a quick mention. modprobe, like insmod, loads a module
into the kernel. It differs in that it will look at the module to be loaded to see
whether it references any symbols that are not currently defined in the kernel. If any
such references are found, modprobe looks for other modules in the current module
search path that define the relevant symbols. When modprobe finds those modules
(which are needed by the module being loaded), it loads them into the kernel as well.
If you use insmod in this situation instead, the command fails with an "unresolved
symbols" message left in the system logfile.
As mentioned before, modules may be removed from the kernel with the rmmod utility.
Note that module removal fails if the kernel believes that the module is still in
use (e.g., a program still has an open file for a device exported by the modules), or if
the kernel has been configured to disallow module removal. It is possible to configure
the kernel to allow "forced" removal of modules, even when they appear to be
busy. If you reach a point where you are considering using this option, however,
things are likely to have gone wrong badly enough that a reboot may well be the better
course of action.
PAGE 26
The lsmod program produces a list of the modules currently loaded in the kernel.
Some other information, such as any other modules making use of a specific module,
is also provided. lsmod works by reading the /proc/modules virtual file. Information
on currently loaded modules can also be found in the sysfs virtual filesystem
under /sys/module.
Version Dependency
Bear in mind that your module's code has to be recompiled for each version of the
kernel that it is linked to--at least, in the absence of modversions, not covered here
as they are more for distribution makers than developers. Modules are strongly tied
to the data structures and function prototypes defined in a particular kernel version;
the interface seen by a module can change significantly from one kernel version to
the next. This is especially true of development kernels, of course.
The kernel does not just assume that a given module has been built against the
proper kernel version. One of the steps in the build process is to link your module
against a file (called vermagic.o) from the current kernel tree; this object contains a
fair amount of information about the kernel the module was built for, including the
target kernel version, compiler version, and the settings of a number of important
configuration variables. When an attempt is made to load a module, this information
can be tested for compatibility with the running kernel. If things don't match,
the module is not loaded; instead, you see something like:
# insmod hello.ko
Error inserting './hello.ko': -1 Invalid module format
A look in the system log file (/var/log/messages or whatever your system is configured
to use) will reveal the specific problem that caused the module to fail to load.
If you need to compile a module for a specific kernel version, you will need to use the
build system and source tree for that particular version. A simple change to the
KERNELDIR variable in the example makefile shown previously does the trick.
Kernel interfaces often change between releases. If you are writing a module that is
intended to work with multiple versions of the kernel (especially if it must work
across major releases), you likely have to make use of macros and #ifdef constructs
to make your code build properly. This edition of this book only concerns itself with
one major version of the kernel, so you do not often see version tests in our example
code. But the need for them does occasionally arise. In such cases, you want to make
use of the definitions found in linux/version.h. This header file defines the following macros:
PAGE 27
UTS_RELEASE
This macro expands to a string describing the version of this kernel tree. For
example, "2.6.10".
LINUX_VERSION_CODE
This macro expands to the binary representation of the kernel version, one byte
for each part of the version release number. For example, the code for 2.6.10 is
132618 (i.e., 0x02060a).* With this information, you can (almost) easily determine
what version of the kernel you are dealing with.
KERNEL_VERSION(major,minor,release)
This is the macro used to build an integer version code from the individual numbers
that build up a version number. For example, KERNEL_VERSION(2,6,10)
expands to 132618. This macro is very useful when you need to compare the
current version and a known checkpoint.
Most dependencies based on the kernel version can be worked around with preprocessor
conditionals by exploiting KERNEL_VERSION and LINUX_VERSION_CODE. Version
dependency should, however, not clutter driver code with hairy #ifdef conditionals;
the best way to deal with incompatibilities is by confining them to a specific header
file. As a general rule, code which is explicitly version (or platform) dependent
should be hidden behind a low-level macro or function. High-level code can then
just call those functions without concern for the low-level details. Code written in
this way tends to be easier to read and more robust.
Platform Dependency
Each computer platform has its peculiarities, and kernel designers are free to exploit
all the peculiarities to achieve better performance in the target object file.
Unlike application developers, who must link their code with precompiled libraries
and stick to conventions on parameter passing, kernel developers can dedicate some
processor registers to specific roles, and they have done so. Moreover, kernel code
can be optimized for a specific processor in a CPU family to get the best from the target
platform: unlike applications that are often distributed in binary format, a custom
compilation of the kernel can be optimized for a specific computer set.
For example, the IA32 (x86) architecture has been subdivided into several different
processor types. The old 80386 processor is still supported (for now), even though
its instruction set is, by modern standards, quite limited. The more modern processors
in this architecture have introduced a number of new capabilities, including
faster instructions for entering the kernel, interprocessor locking, copying data, etc.
Newer processors can also, when operated in the correct mode, employ 36-bit (or
* This allows up to 256 development versions between stable versions.
PAGE 28
larger) physical addresses, allowing them to address more than 4 GB of physical
memory. Other processor families have seen similar improvements. The kernel,
depending on various configuration options, can be built to make use of these additional
features.
Clearly, if a module is to work with a given kernel, it must be built with the same
understanding of the target processor as that kernel was. Once again, the vermagic.o
object comes in to play. When a module is loaded, the kernel checks the processor specific
configuration options for the module and makes sure they match the running
kernel. If the module was compiled with different options, it is not loaded.
If you are planning to write a driver for general distribution, you may well be wondering
just how you can possibly support all these different variations. The best
answer, of course, is to release your driver under a GPL-compatible license and contribute
it to the mainline kernel. Failing that, distributing your driver in source form
and a set of scripts to compile it on the user's system may be the best answer. Some
vendors have released tools to make this task easier. If you must distribute your
driver in binary form, you need to look at the different kernels provided by your target
distributions, and provide a version of the module for each. Be sure to take into
account any errata kernels that may have been released since the distribution was
produced. Then, there are licensing issues to be considered, as we discussed in the
section "License Terms" in Chapter 1. As a general rule, distributing things in source
form is an easier way to make your way in the world.
The Kernel Symbol Table
We've seen how insmod resolves undefined symbols against the table of public kernel
symbols. The table contains the addresses of global kernel items--functions and
variables--that are needed to implement modularized drivers. When a module is
loaded, any symbol exported by the module becomes part of the kernel symbol table.
In the usual case, a module implements its own functionality without the need to
export any symbols at all. You need to export symbols, however, whenever other
modules may benefit from using them.
New modules can use symbols exported by your module, and you can stack new
modules on top of other modules. Module stacking is implemented in the mainstream
kernel sources as well: the msdos filesystem relies on symbols exported by the
fat module, and each input USB device module stacks on the usbcore and input modules.
Module stacking is useful in complex projects. If a new abstraction is implemented in
the form of a device driver, it might offer a plug for hardware-specific implementations.
For example, the video-for-linux set of drivers is split into a generic module that
exports symbols used by lower-level device drivers for specific hardware. According to
your setup, you load the generic video module and the specific module for your
installed hardware. Support for parallel ports and the wide variety of attachable
PAGE 29
devices is handled in the same way, as is the USB kernel subsystem. Stacking in the
parallel port subsystem is shown in Figure 2-2; the arrows show the communications
between the modules and with the kernel programming interface.
Figure 2-2. Stacking of parallel port driver modules
When using stacked modules, it is helpful to be aware of the modprobe utility. As we
described earlier, modprobe functions in much the same way as insmod, but it also
loads any other modules that are required by the module you want to load. Thus,
one modprobe command can sometimes replace several invocations of insmod
(although you'll still need insmod when loading your own modules from the current
directory, because modprobe looks only in the standard installed module directories).
Using stacking to split modules into multiple layers can help reduce development
time by simplifying each layer. This is similar to the separation between mechanism
and policy that we discussed in Chapter 1.
The Linux kernel header files provide a convenient way to manage the visibility of
your symbols, thus reducing namespace pollution (filling the namespace with names
that may conflict with those defined elsewhere in the kernel) and promoting proper
information hiding. If your module needs to export symbols for other modules to
use, the following macros should be used.
EXPORT_SYMBOL(name);
EXPORT_SYMBOL_GPL(name);
Either of the above macros makes the given symbol available outside the module.
The _GPL version makes the symbol available to GPL-licensed modules only. Symbols
must be exported in the global part of the module's file, outside of any function,
because the macros expand to the declaration of a special-purpose variable that
is expected to be accessible globally. This variable is stored in a special part of the
module executable (an "ELF section") that is used by the kernel at load time to find
the variables exported by the module. (Interested readers can look at <linux/module.h>
for the details, even though the details are not needed to make things work.)
PAGE 30
Preliminaries
We are getting closer to looking at some actual module code. But first, we need to
look at some other things that need to appear in your module source files. The kernel
is a unique environment, and it imposes its own requirements on code that
would interface with it.
Most kernel code ends up including a fairly large number of header files to get definitions
of functions, data types, and variables. We'll examine these files as we come to
them, but there are a few that are specific to modules, and must appear in every loadable
module. Thus, just about all module code has the following:
#include <linux/module.h>
#include <linux/init.h>
module.h contains a great many definitions of symbols and functions needed by loadable
modules. You need init.h to specify your initialization and cleanup functions, as
we saw in the "hello world" example above, and which we revisit in the next section.
Most modules also include moduleparam.h to enable the passing of parameters
to the module at load time; we will get to that shortly.
It is not strictly necessary, but your module really should specify which license
applies to its code. Doing so is just a matter of including a MODULE_LICENSE line:
MODULE_LICENSE("GPL");
The specific licenses recognized by the kernel are "GPL" (for any version of the GNU
General Public License), "GPL v2" (for GPL version two only), "GPL and additional
rights," "Dual BSD/GPL," "Dual MPL/GPL," and "Proprietary." Unless your module
is explicitly marked as being under a free license recognized by the kernel, it is
assumed to be proprietary, and the kernel is "tainted" when the module is loaded. As
we mentioned in the section "License Terms" in Chapter 1, kernel developers tend to
be unenthusiastic about helping users who experience problems after loading proprietary
modules.
Other descriptive definitions that can be contained within a module include
MODULE_AUTHOR (stating who wrote the module), MODULE_DESCRIPTION (a human-readable
statement of what the module does), MODULE_VERSION (for a code revision number;
see the comments in <linux/module.h> for the conventions to use in creating
version strings), MODULE_ALIAS (another name by which this module can be known),
and MODULE_DEVICE_TABLE (to tell user space about which devices the module supports).
We'll discuss MODULE_ALIAS in Chapter 11 and MODULE_DEVICE_TABLE in
Chapter 12.
The various MODULE_ declarations can appear anywhere within your source file outside
of a function. A relatively recent convention in kernel code, however, is to put
these declarations at the end of the file.
PAGE 31
Initialization and Shutdown
As already mentioned, the module initialization function registers any facility offered
by the module. By facility, we mean a new functionality, be it a whole driver or a new
software abstraction, that can be accessed by an application. The actual definition of
the initialization function always looks like:
static int __init initialization_function(void)
{
/* Initialization code here */
}
module_init(initialization_function);
Initialization functions should be declared static, since they are not meant to be visible
outside the specific file; there is no hard rule about this, though, as no function is
exported to the rest of the kernel unless explicitly requested. The __init token in the
definition may look a little strange; it is a hint to the kernel that the given function is
used only at initialization time. The module loader drops the initialization function
after the module is loaded, making its memory available for other uses. There is
a similar tag (__initdata) for data used only during initialization. Use of __init and
__initdata is optional, but it is worth the trouble. Just be sure not to use them for
any function (or data structure) you will be using after initialization completes. You
may also encounter __devinit and __devinitdata in the kernel source; these translate
to __init and __initdata only if the kernel has not been configured for hotpluggable
devices. We will look at hotplug support in Chapter 14.
The use of module_init is mandatory. This macro adds a special section to the module's
object code stating where the module's initialization function is to be found.
Without this definition, your initialization function is never called.
Modules can register many different types of facilities, including different kinds of
devices, filesystems, cryptographic transforms, and more. For each facility, there is a
specific kernel function that accomplishes this registration. The arguments passed to
the kernel registration functions are usually pointers to data structures describing the
new facility and the name of the facility being registered. The data structure usually
contains pointers to module functions, which is how functions in the module body
get called.
The items that can be registered go beyond the list of device types mentioned in
Chapter 1. They include, among others, serial ports, miscellaneous devices, sysfs
entries, /proc files, executable domains, and line disciplines. Many of those registrable
items support functions that aren't directly related to hardware but remain in the
"software abstractions" field. Those items can be registered, because they are integrated
into the driver's functionality anyway (like /proc files and line disciplines for
example).
PAGE 32
There are other facilities that can be registered as add-ons for certain drivers, but
their use is so specific that it's not worth talking about them; they use the stacking
technique, as described in the section "The Kernel Symbol Table." If you want to
probe further, you can grep for EXPORT_SYMBOL in the kernel sources, and find the
entry points offered by different drivers. Most registration functions are prefixed with
register_, so another possible way to find them is to grep for register_ in the kernel
source.
The Cleanup Function
Every nontrivial module also requires a cleanup function, which unregisters interfaces
and returns all resources to the system before the module is removed. This
function is defined as:
static void __exit cleanup_function(void)
{
/* Cleanup code here */
}
module_exit(cleanup_function);
The cleanup function has no value to return, so it is declared void. The __exit modifier
marks the code as being for module unload only (by causing the compiler to
place it in a special ELF section). If your module is built directly into the kernel,
or if your kernel is configured to disallow the unloading of modules, functions
marked __exit are simply discarded. For this reason, a function marked __exit can
be called only at module unload or system shutdown time; any other use is an error.
Once again, the module_exit declaration is necessary to enable the kernel to find your
cleanup function.
If your module does not define a cleanup function, the kernel does not allow it to be
unloaded.
Error Handling During Initialization
One thing you must always bear in mind when registering facilities with the kernel
is that the registration could fail. Even the simplest action often requires memory
allocation, and the required memory may not be available. So module code must
always check return values, and be sure that the requested operations have actually
succeeded.
If any errors occur when you register utilities, the first order of business is to decide
whether the module can continue initializing itself anyway. Often, the module can
continue to operate after a registration failure, with degraded functionality if necessary.
Whenever possible, your module should press forward and provide what capabilities
it can after things fail.
PAGE 33
If it turns out that your module simply cannot load after a particular type of failure,
you must undo any registration activities performed before the failure. Linux doesn't
keep a per-module registry of facilities that have been registered, so the module must
back out of everything itself if initialization fails at some point. If you ever fail to
unregister what you obtained, the kernel is left in an unstable state; it contains internal
pointers to code that no longer exists. In such situations, the only recourse, usually,
is to reboot the system. You really do want to take care to do the right thing
when an initialization error occurs.
Error recovery is sometimes best handled with the goto statement. We normally hate
to use goto, but in our opinion, this is one situation where it is useful. Careful use of
goto in error situations can eliminate a great deal of complicated, highly-indented,
"structured" logic. Thus, in the kernel, goto is often used as shown here to deal with
errors.
The following sample code (using fictitious registration and unregistration functions)
behaves correctly if initialization fails at any point:
int __init my_init_function(void)
{
int err;
/* registration takes a pointer and a name */
err = register_this(ptr1, "skull");
if (err) goto fail_this;
err = register_that(ptr2, "skull");
if (err) goto fail_that;
err = register_those(ptr3, "skull");
if (err) goto fail_those;
return 0; /* success */
fail_those: unregister_that(ptr2, "skull");
fail_that: unregister_this(ptr1, "skull");
fail_this: return err; /* propagate the error */
}
This code attempts to register three (fictitious) facilities. The goto statement is used
in case of failure to cause the unregistration of only the facilities that had been successfully
registered before things went bad.
Another option, requiring no hairy goto statements, is keeping track of what has
been successfully registered and calling your module's cleanup function in case of
any error. The cleanup function unrolls only the steps that have been successfully
accomplished. This alternative, however, requires more code and more CPU time, so
in fast paths you still resort to goto as the best error-recovery tool.
The return value of my_init_function, err, is an error code. In the Linux kernel, error
codes are negative numbers belonging to the set defined in <linux/errno.h>. If you
want to generate your own error codes instead of returning what you get from other
PAGE 34
functions, you should include <linux/errno.h> in order to use symbolic values such
as -ENODEV, -ENOMEM, and so on. It is always good practice to return appropriate error
codes, because user programs can turn them to meaningful strings using perror or
similar means.
Obviously, the module cleanup function must undo any registration performed by
the initialization function, and it is customary (but not usually mandatory) to unregister
facilities in the reverse order used to register them:
void __exit my_cleanup_function(void)
{
unregister_those(ptr3, "skull");
unregister_that(ptr2, "skull");
unregister_this(ptr1, "skull");
return;
}
If your initialization and cleanup are more complex than dealing with a few items,
the goto approach may become difficult to manage, because all the cleanup code
must be repeated within the initialization function, with several labels intermixed.
Sometimes, therefore, a different layout of the code proves more successful.
What you'd do to minimize code duplication and keep everything streamlined is to
call the cleanup function from within the initialization whenever an error occurs.
The cleanup function then must check the status of each item before undoing its registration.
In its simplest form, the code looks like the following:
struct something *item1;
struct somethingelse *item2;
int stuff_ok;
void my_cleanup(void)
{
if (item1)
release_thing(item1);
if (item2)
release_thing2(item2);
if (stuff_ok)
unregister_stuff( );
return;
}
int __init my_init(void)
{
int err = -ENOMEM;
item1 = allocate_thing(arguments);
item2 = allocate_thing2(arguments2);
if (!item1 || !item2)
goto fail;
err = register_stuff(item1, item2);
if (!err)
PAGE 35
stuff_ok = 1;
else
goto fail;
return 0; /* success */
fail:
my_cleanup( );
return err;
}
As shown in this code, you may or may not need external flags to mark success of the
initialization step, depending on the semantics of the registration/allocation function
you call. Whether or not flags are needed, this kind of initialization scales well to a
large number of items and is often better than the technique shown earlier. Note,
however, that the cleanup function cannot be marked __exit when it is called by
nonexit code, as in the previous example.
Module-Loading Races
Thus far, our discussion has skated over an important aspect of module loading: race
conditions. If you are not careful in how you write your initialization function, you
can create situations that can compromise the stability of the system as a whole. We
will discuss race conditions later in this book; for now, a couple of quick points will
have to suffice.
The first is that you should always remember that some other part of the kernel can
make use of any facility you register immediately after that registration has completed.
It is entirely possible, in other words, that the kernel will make calls into your
module while your initialization function is still running. So your code must be prepared
to be called as soon as it completes its first registration. Do not register any
facility until all of your internal initialization needed to support that facility has been
completed.
You must also consider what happens if your initialization function decides to fail,
but some part of the kernel is already making use of a facility your module has registered.
If this situation is possible for your module, you should seriously consider not
failing the initialization at all. After all, the module has clearly succeeded in exporting
something useful. If initialization must fail, it must carefully step around any possible
operations going on elsewhere in the kernel until those operations have
completed.
Module Parameters
Several parameters that a driver needs to know can change from system to system.
These can vary from the device number to use (as we'll see in the next chapter) to
numerous aspects of how the driver should operate. For example, drivers for SCSI
PAGE 36
adapters often have options controlling the use of tagged command queuing, and the
Integrated Device Electronics (IDE) drivers allow user control of DMA operations. If
your driver controls older hardware, it may also need to be told explicitly where to
find that hardware's I/O ports or I/O memory addresses. The kernel supports these
needs by making it possible for a driver to designate parameters that may be changed
when the driver's module is loaded.
These parameter values can be assigned at load time by insmod or modprobe; the latter
can also read parameter assignment from its configuration file (/etc/modprobe.
conf). The commands accept the specification of several types of values on the command
line. As a way of demonstrating this capability, imagine a much-needed
enhancement to the "hello world" module (called hellop) shown at the beginning of
this chapter. We add two parameters: an integer value called howmany and a character
string called whom. Our vastly more functional module then, at load time, greets whom
not just once, but howmany times. Such a module could then be loaded with a command
line such as:
insmod hellop howmany=10 whom="Mom"
Upon being loaded that way, hellop would say "Hello, Mom" 10 times.
However, before insmod can change module parameters, the module must make
them available. Parameters are declared with the module_param macro, which is
defined in moduleparam.h. module_param takes three parameters: the name of the
variable, its type, and a permissions mask to be used for an accompanying sysfs
entry. The macro should be placed outside of any function and is typically found
near the head of the source file. So hellop would declare its parameters and make
them available to insmod as follows:
static char *whom = "world";
static int howmany = 1;
module_param(howmany, int, S_IRUGO);
module_param(whom, charp, S_IRUGO);
Numerous types are supported for module parameters:
bool
invbool
A boolean (true or false) value (the associated variable should be of type int).
The invbool type inverts the value, so that true values become false and vice
versa.
charp
A char pointer value. Memory is allocated for user-provided strings, and the
pointer is set accordingly.
PAGE 37
int
long
short
uint
ulong
ushort
Basic integer values of various lengths. The versions starting with u are for
unsigned values.
Array parameters, where the values are supplied as a comma-separated list, are also
supported by the module loader. To declare an array parameter, use:
module_param_array(name,type,nump,perm);
Where name is the name of your array (and of the parameter), type is the type of the
array elements, nump is a pointer to an integer variable, and perm is the usual permissions value. If
the array parameter is set at load time, nump is set to the number of values supplied.
The module loader refuses to accept more values than will fit in the array.
If you really need a type that does not appear in the list above, there are hooks in the
module code that allow you to define them; see moduleparam.h for details on how to
do that. All module parameters should be given a default value; insmod changes the
value only if explicitly told to by the user. The module can check for explicit parameters
by testing parameters against their default values.
The final module_param field is a permission value; you should use the definitions
found in <linux/stat.h>. This value controls who can access the representation of the
module parameter in sysfs. If perm is set to 0, there is no sysfs entry at all; otherwise,
it appears under /sys/module* with the given set of permissions. Use S_IRUGO for a
parameter that can be read by the world but cannot be changed; S_IRUGO|S_IWUSR
allows root to change the parameter. Note that if a parameter is changed by sysfs, the
value of that parameter as seen by your module changes, but your module is not
notified in any other way. You should probably not make module parameters writable,
unless you are prepared to detect the change and react accordingly.
Doing It in User Space
A Unix programmer who's addressing kernel issues for the first time might be nervous
about writing a module. Writing a user program that reads and writes directly
to the device ports may be easier.
Indeed, there are some arguments in favor of user-space programming, and sometimes
writing a so-called user-space device driver is a wise alternative to kernel hacking.
In this section, we discuss some of the reasons why you might write a driver in
* As of this writing, there is talk of moving parameters elsewhere within sysfs, however.
PAGE 38
user space. This book is about kernel-space drivers, however, so we do not go
beyond this introductory discussion.
The advantages of user-space drivers are:
- The full C library can be linked in. The driver can perform many exotic tasks without resorting to external programs (the utility programs implementing usage policies that are usually distributed along with the driver itself).
- The programmer can run a conventional debugger on the driver code without having to go through contortions to debug a running kernel.
- If a user-space driver hangs, you can simply kill it. Problems with the driver are unlikely to hang the entire system, unless the hardware being controlled is really misbehaving.
- User memory is swappable, unlike kernel memory. An infrequently used device with a huge driver won't occupy RAM that other programs could be using, except when it is actually in use.
- A well-designed driver program can still, like kernel-space drivers, allow concurrent access to a device.
- If you must write a closed-source driver, the user-space option makes it easier for you to avoid ambiguous licensing situations and problems with changing kernel interfaces.
For example, USB drivers can be written for user space; see the (still young) libusb
project at libusb.sourceforge.net and "gadgetfs" in the kernel source. Another example
is the X server: it knows exactly what the hardware can do and what it can't, and
it offers the graphic resources to all X clients. Note, however, that there is a slow but
steady drift toward frame-buffer-based graphics environments, where the X server
acts only as a server based on a real kernel-space device driver for actual graphic
manipulation.
Usually, the writer of a user-space driver implements a server process, taking over
from the kernel the task of being the single agent in charge of hardware control. Client
applications can then connect to the server to perform actual communication
with the device; therefore, a smart driver process can allow concurrent access to the
device. This is exactly how the X server works.
But the user-space approach to device driving has a number of drawbacks. The most
important are:
- Interrupts are not available in user space. There are work-arounds for this limitation on some platforms, such as the vm86 system call on the IA32 architecture.
- Direct access to memory is possible only by mmapping /dev/mem, and only a privileged user can do that.
- Access to I/O ports is available only after calling ioperm or iopl. Moreover, not all platforms support these system calls, and access to /dev/port can be too slow
PAGE 39
to be effective. Both the system calls and the device file are reserved to a privileged user.
- Response time is slower, because a context switch is required to transfer information or actions between the client and the hardware.
- Worse yet, if the driver has been swapped to disk, response time is unacceptably long. Using the mlock system call might help, but usually you'll need to lock many memory pages, because a user-space program depends on a lot of library code. mlock, too, is limited to privileged users.
- The most important devices can't be handled in user space, including, but not limited to, network interfaces and block devices.
As you see, user-space drivers can't do that much after all. Interesting applications
nonetheless exist: for example, support for SCSI scanner devices (implemented by
the SANE package) and CD writers (implemented by cdrecord and other tools). In
both cases, user-level device drivers rely on the "SCSI generic" kernel driver, which
exports low-level SCSI functionality to user-space programs so they can drive their
own hardware.
One case in which working in user space might make sense is when you are beginning
to deal with new and unusual hardware. This way you can learn to manage your
hardware without the risk of hanging the whole system. Once you've done that,
encapsulating the software in a kernel module should be a painless operation.
Quick Reference
This section summarizes the kernel functions, variables, macros, and /proc files that
we've touched on in this chapter. It is meant to act as a reference. Each item is listed
after the relevant header file, if any. A similar section appears at the end of almost
every chapter from here on, summarizing the new symbols introduced in the chapter.
Entries in this section generally appear in the same order in which they were
introduced in the chapter:
insmod
modprobe
rmmod
User-space utilities that load modules into the running kernels and remove them.
#include <linux/init.h>
module_init(init_function);
module_exit(cleanup_function);
Macros that designate a module's initialization and cleanup functions.
PAGE 40
__init
__initdata
__exit
__exitdata
Markers for functions (__init and __exit) and data (__initdata and __exitdata)
that are only used at module initialization or cleanup time. Items marked for initialization
may be discarded once initialization completes; the exit items may be
discarded if module unloading has not been configured into the kernel. These
markers work by causing the relevant objects to be placed in a special ELF section
in the executable file.
#include <linux/sched.h>
One of the most important header files. This file contains definitions of much of
the kernel API used by the driver, including functions for sleeping and numerous
variable declarations.
struct task_struct *current;
The current process.
current->pid
current->comm
The process ID and command name for the current process.
obj-m
A makefile symbol used by the kernel build system to determine which modules
should be built in the current directory.
/sys/module
/proc/modules
/sys/module is a sysfs directory hierarchy containing information on currently loaded
modules. /proc/modules is the older, single-file version of that information.
Entries contain the module name, the amount of memory each module
occupies, and the usage count. Extra strings are appended to each line to specify
flags that are currently active for the module.
vermagic.o
An object file from the kernel source directory that describes the environment a
module was built for.
#include <linux/module.h>
Required header. It must be included by a module source.
#include <linux/version.h>
A header file containing information on the version of the kernel being built.
LINUX_VERSION_CODE
Integer macro, useful to #ifdef version dependencies.
PAGE 41
EXPORT_SYMBOL (symbol);
EXPORT_SYMBOL_GPL (symbol);
Macro used to export a symbol to the kernel. The second form exports without
using versioning information, and the third limits the export to GPL-licensed
modules.
MODULE_AUTHOR(author);
MODULE_DESCRIPTION(description);
MODULE_VERSION(version_string);
MODULE_DEVICE_TABLE(table_info);
MODULE_LICENSE(license);
MODULE_ALIAS(alternate_name);
Place documentation on the module in the object file.
module_init(init_function);
module_exit(exit_function);
Macros that declare a module's initialization and cleanup functions.
#include <linux/moduleparam.h>
module_param(variable, type, perm);
Macro that creates a module parameter that can be adjusted by the user when
the module is loaded (or at boot time for built-in code). The type can be one of
bool, charp, int, invbool, long, short, ushort, uint, ulong, or intarray.
#include <linux/kernel.h>
int printk(const char * fmt, ...);
The analogue of printf for kernel code.
PAGE 42
CHAPTER 3
Char Drivers
The goal of this chapter is to write a complete char device driver. We develop a character
driver because this class is suitable for most simple hardware devices. Char
drivers are also easier to understand than block drivers or network drivers (which we
get to in later chapters). Our ultimate aim is to write a modularized char driver, but
we won't talk about modularization issues in this chapter.
Throughout the chapter, we present code fragments extracted from a real device
driver: scull (Simple Character Utility for Loading Localities). scull is a char driver
that acts on a memory area as though it were a device. In this chapter, because of
that peculiarity of scull, we use the word device interchangeably with "the memory
area used by scull."
The advantage of scull is that it isn't hardware dependent. scull just acts on some
memory, allocated from the kernel. Anyone can compile and run scull, and scull is
portable across the computer architectures on which Linux runs. On the other hand,
the device doesn't do anything "useful" other than demonstrate the interface
between the kernel and char drivers and allow the user to run some tests.
The Design of scull
The first step of driver writing is defining the capabilities (the mechanism) the driver
will offer to user programs. Since our "device" is part of the computer's memory,
we're free to do what we want with it. It can be a sequential or random-access
device, one device or many, and so on.
To make scull useful as a template for writing real drivers for real devices, we'll show
you how to implement several device abstractions on top of the computer memory,
each with a different personality.
The scull source implements the following devices. Each kind of device implemented
by the module is referred to as a type.
PAGE 43
scull0 to scull3
Four devices, each consisting of a memory area that is both global and persistent.
Global means that if the device is opened multiple times, the data contained
within the device is shared by all the file descriptors that opened it.
Persistent means that if the device is closed and reopened, data isn't lost. This
device can be fun to work with, because it can be accessed and tested using conventional
commands, such as cp, cat, and shell I/O redirection.
scullpipe0 to scullpipe3
Four FIFO (first-in-first-out) devices, which act like pipes. One process reads
what another process writes. If multiple processes read the same device, they
contend for data. The internals of scullpipe will show how blocking and nonblocking
read and write can be implemented without having to resort to interrupts.
Although real drivers synchronize with their devices using hardware
interrupts, the topic of blocking and nonblocking operations is an important one
and is separate from interrupt handling (covered in Chapter 10).
scullsingle
scullpriv
sculluid
scullwuid
These devices are similar to scull0 but with some limitations on when an open is
permitted. The first (scullsingle) allows only one process at a time to use the
driver, whereas scullpriv is private to each virtual console (or X terminal session),
because processes on each console/terminal get different memory areas.
sculluid and scullwuid can be opened multiple times, but only by one user at a
time; the former returns an error of "Device Busy" if another user is locking the
device, whereas the latter implements blocking open. These variations of scull
would appear to be confusing policy and mechanism, but they are worth looking
at, because some real-life devices require this sort of management.
Each of the scull devices demonstrates different features of a driver and presents different
difficulties. This chapter covers the internals of scull0 to scull3; the more
advanced devices are covered in Chapter 6. scullpipe is described in the section "A
Blocking I/O Example," and the others are described in "Access Control on a Device
File."
Major and Minor Numbers
Char devices are accessed through names in the filesystem. Those names are called
special files or device files or simply nodes of the filesystem tree; they are conventionally
located in the /dev directory. Special files for char drivers are identified by a "c"
in the first column of the output of ls -l. Block devices appear in /dev as well, but
they are identified by a "b." The focus of this chapter is on char devices, but much of
the following information applies to block devices as well.
PAGE 44
If you issue the ls -l command, you'll see two numbers (separated by a comma) in
the device file entries before the date of the last modification, where the file length
normally appears. These numbers are the major and minor device number for the
particular device. The following listing shows a few devices as they appear on a typical
system. Their major numbers are 1, 4, 7, and 10, while the minors are 1, 3, 5, 64,
65, and 129.
crw-rw-rw- 1 root root 1, 3 Apr 11 2002 null
crw------- 1 root root 10, 1 Apr 11 2002 psaux
crw------- 1 root root 4, 1 Oct 28 03:04 tty1
crw-rw-rw- 1 root tty 4, 64 Apr 11 2002 ttys0
crw-rw---- 1 root uucp 4, 65 Apr 11 2002 ttyS1
crw--w---- 1 vcsa tty 7, 1 Apr 11 2002 vcs1
crw--w---- 1 vcsa tty 7, 129 Apr 11 2002 vcsa1
crw-rw-rw- 1 root root 1, 5 Apr 11 2002 zero
Traditionally, the major number identifies the driver associated with the device. For
example, /dev/null and /dev/zero are both managed by driver 1, whereas virtual consoles
and serial terminals are managed by driver 4; similarly, both vcs1 and vcsa1
devices are managed by driver 7. Modern Linux kernels allow multiple drivers to
share major numbers, but most devices that you will see are still organized on the
one-major-one-driver principle.
The minor number is used by the kernel to determine exactly which device is being
referred to. Depending on how your driver is written (as we will see below), you can
either get a direct pointer to your device from the kernel, or you can use the minor
number yourself as an index into a local array of devices. Either way, the kernel itself
knows almost nothing about minor numbers beyond the fact that they refer to
devices implemented by your driver.
The Internal Representation of Device Numbers
Within the kernel, the dev_t type (defined in <linux/types.h>) is used to hold device
numbers--both the major and minor parts. As of Version 2.6.0 of the kernel, dev_t is
a 32-bit quantity with 12 bits set aside for the major number and 20 for the minor
number. Your code should, of course, never make any assumptions about the internal
organization of device numbers; it should, instead, make use of a set of macros
found in <linux/kdev_t.h>. To obtain the major or minor parts of a dev_t, use:
MAJOR(dev_t dev);
MINOR(dev_t dev);
If, instead, you have the major and minor numbers and need to turn them into a dev_t,
use:
MKDEV(int major, int minor);
Note that the 2.6 kernel can accommodate a vast number of devices, while previous
kernel versions were limited to 255 major and 255 minor numbers. One assumes
PAGE 45
that the wider range will be sufficient for quite some time, but the computing field is
littered with erroneous assumptions of that nature. So you should expect that the
format of dev_t could change again in the future; if you write your drivers carefully,
however, these changes will not be a problem.
Allocating and Freeing Device Numbers
One of the first things your driver will need to do when setting up a char device is to
obtain one or more device numbers to work with. The necessary function for this
task is register_chrdev_region, which is declared in <linux/fs.h>:
int register_chrdev_region(dev_t first, unsigned int count,
char *name);
Here, first is the beginning device number of the range you would like to allocate.
The minor number portion of first is often 0, but there is no requirement to that
effect. count is the total number of contiguous device numbers you are requesting.
Note that, if count is large, the range you request could spill over to the next major
number; but everything will still work properly as long as the number range you
request is available. Finally, name is the name of the device that should be associated
with this number range; it will appear in /proc/devices and sysfs.
As with most kernel functions, the return value from register_chrdev_region will be 0
if the allocation was successfully performed. In case of error, a negative error code
will be returned, and you will not have access to the requested region.
register_chrdev_region works well if you know ahead of time exactly which device
numbers you want. Often, however, you will not know which major numbers your
device will use; there is a constant effort within the Linux kernel development community
to move over to the use of dynamically-allocated device numbers. The kernel
will happily allocate a major number for you on the fly, but you must request this
allocation by using a different function:
int alloc_chrdev_region(dev_t *dev, unsigned int firstminor,
unsigned int count, char *name);
With this function, dev is an output-only parameter that will, on successful completion,
hold the first number in your allocated range. firstminor should be the
requested first minor number to use; it is usually 0. The count and name parameters
work like those given to register_chrdev_region.
Regardless of how you allocate your device numbers, you should free them when
they are no longer in use. Device numbers are freed with:
void unregister_chrdev_region(dev_t first, unsigned int count);
The usual place to call unregister_chrdev_region would be in your module's cleanup
function.
PAGE 46
The above functions allocate device numbers for your driver's use, but they do not
tell the kernel anything about what you will actually do with those numbers. Before a
user-space program can access one of those device numbers, your driver needs to
connect them to its internal functions that implement the device's operations. We
will describe how this connection is accomplished shortly, but there are a couple of
necessary digressions to take care of first.
Dynamic Allocation of Major Numbers
Some major device numbers are statically assigned to the most common devices. A
list of those devices can be found in Documentation/devices.txt within the kernel
source tree. The chances of a static number having already been assigned for the use
of your new driver are small, however, and new numbers are not being assigned. So,
as a driver writer, you have a choice: you can simply pick a number that appears to
be unused, or you can allocate major numbers in a dynamic manner. Picking a number
may work as long as the only user of your driver is you; once your driver is more
widely deployed, a randomly picked major number will lead to conflicts and trouble.
Thus, for new drivers, we strongly suggest that you use dynamic allocation to obtain
your major device number, rather than choosing a number randomly from the ones
that are currently free. In other words, your drivers should almost certainly be using
alloc_chrdev_region rather than register_chrdev_region.
The disadvantage of dynamic assignment is that you can't create the device nodes in
advance, because the major number assigned to your module will vary. For normal
use of the driver, this is hardly a problem, because once the number has been
assigned, you can read it from /proc/devices.*
To load a driver using a dynamic major number, therefore, the invocation of insmod
can be replaced by a simple script that, after calling insmod, reads /proc/devices in
order to create the special file(s).
A typical /proc/devices file looks like the following:
Character devices:
1 mem
2 pty
3 ttyp
4 ttyS
6 lp
7 vcs
10 misc
13 input
14 sound
* Even better device information can usually be obtained from sysfs, generally mounted on /sys on 2.6-based systems. Getting scull to export information via sysfs is beyond the scope of this chapter, however; we'll return to this topic in Chapter 14.
PAGE 47
21 sg
180 usb
Block devices:
2 fd
8 sd
11 sr
65 sd
66 sd
The script to load a module that has been assigned a dynamic number can, therefore,
be written using a tool such as awk to retrieve information from /proc/devices in
order to create the files in /dev.
The following script, scull_load, is part of the scull distribution. The user of a driver
that is distributed in the form of a module can invoke such a script from the system's
rc.local file or call it manually whenever the module is needed.
#!/bin/sh
module="scull"
device="scull"
mode="664"
# invoke insmod with all arguments we got
# and use a pathname, as newer modutils don't look in . by default
/sbin/insmod ./$module.ko $* || exit 1
# remove stale nodes
rm -f /dev/${device}[0-3]
major=$(awk "\$2= =\"$module\" {print \$1}" /proc/devices)
mknod /dev/${device}0 c $major 0
mknod /dev/${device}1 c $major 1
mknod /dev/${device}2 c $major 2
mknod /dev/${device}3 c $major 3
# give appropriate group/permissions, and change the group.
# Not all distributions have staff, some have "wheel" instead.
group="staff"
grep -q '^staff:' /etc/group || group="wheel"
chgrp $group /dev/${device}[0-3]
chmod $mode /dev/${device}[0-3]
The script can be adapted for another driver by redefining the variables and adjusting
the mknod lines. The script just shown creates four devices because four is the
default in the scull sources.
The last few lines of the script may seem obscure: why change the group and mode
of a device? The reason is that the script must be run by the superuser, so newly created
special files are owned by root. The permission bits default so that only root has
write access, while anyone can get read access. Normally, a device node requires a
PAGE 48
different access policy, so in some way or another access rights must be changed.
The default in our script is to give access to a group of users, but your needs may
vary. In the section "Access Control on a Device File" in Chapter 6, the code for sculluid
demonstrates how the driver can enforce its own kind of authorization for device
access.
A scull_unload script is also available to clean up the /dev directory and remove the
module.
As an alternative to using a pair of scripts for loading and unloading, you could write
an init script, ready to be placed in the directory your distribution uses for these
scripts.* As part of the scull source, we offer a fairly complete and configurable example
of an init script, called scull.init; it accepts the conventional arguments--start,
stop, and restart--and performs the role of both scull_load and scull_unload.
If repeatedly creating and destroying /dev nodes sounds like overkill, there is a useful
work-around. If you are loading and unloading only a single driver, you can just use
rmmod and insmod after the first time you create the special files with your script:
dynamic numbers are not randomized, and you can count on the same number
being chosen each time if you don't load any other (dynamic) modules. Avoiding
lengthy scripts is useful during development. But this trick, clearly, doesn't scale to
more than one driver at a time.
The best way to assign major numbers, in our opinion, is by defaulting to dynamic
allocation while leaving yourself the option of specifying the major number at load
time, or even at compile time. The scull implementation works in this way; it uses a
global variable, scull_major, to hold the chosen number (there is also a scull_minor
for the minor number). The variable is initialized to SCULL_MAJOR, defined in scull.h.
The default value of SCULL_MAJOR in the distributed source is 0, which means "use
dynamic assignment." The user can accept the default or choose a particular major
number, either by modifying the macro before compiling or by specifying a value for
scull_major on the insmod command line. Finally, by using the scull_load script, the
user can pass arguments to insmod on scull_load's command line.
Here's the code we use in scull's source to get a major number:
if (scull_major) {
dev = MKDEV(scull_major, scull_minor);
result = register_chrdev_region(dev, scull_nr_devs, "scull");
} else {
result = alloc_chrdev_region(&dev, scull_minor, scull_nr_devs,
* The Linux Standard Base specifies that init scripts should be placed in /etc/init.d, but some distributions
still place them elsewhere. In addition, if your script is to be run at boot time, you need to make a link to it from
the appropriate run-level directory (i.e., .../rc3.d).
Though certain kernel developers have threatened to do exactly that in the future.
The init script scull.init doesn't accept driver options on the command line, but it supports a configuration
file, because it's designed for automatic use at boot and shutdown time.
PAGE 49
"scull");
scull_major = MAJOR(dev);
}
if (result < 0) {
printk(KERN_WARNING "scull: can't get major %d\n", scull_major);
return result;
}
Almost all of the sample drivers used in this book use similar code for their major
number assignment.
Some Important Data Structures
As you can imagine, device number registration is just the first of many tasks that
driver code must carry out. We will soon look at other important driver components,
but one other digression is needed first. Most of the fundamental driver operations
involve three important kernel data structures, called file_operations, file,
and inode. A basic familiarity with these structures is required to be able to do much
of anything interesting, so we will now take a quick look at each of them before getting
into the details of how to implement the fundamental driver operations.
File Operations
So far, we have reserved some device numbers for our use, but we have not yet connected
any of our driver's operations to those numbers. The file_operations structure
is how a char driver sets up this connection. The structure, defined in <linux/fs.h>,
is a collection of function pointers. Each open file (represented internally by a file
structure, which we will examine shortly) is associated with its own set of functions
(by including a field called f_op that points to a file_operations structure). The
operations are mostly in charge of implementing the system calls and are therefore,
named open, read, and so on. We can consider the file to be an "object" and the
functions operating on it to be its "methods," using object-oriented programming
terminology to denote actions declared by an object to act on itself. This is the first
sign of object-oriented programming we see in the Linux kernel, and we'll see more
in later chapters.
Conventionally, a file_operations structure or a pointer to one is called fops (or
some variation thereof). Each field in the structure must point to the function in the
driver that implements a specific operation, or be left NULL for unsupported operations.
The exact behavior of the kernel when a NULL pointer is specified is different
for each function, as the list later in this section shows.
The following list introduces all the operations that an application can invoke on a
device. We've tried to keep the list brief so it can be used as a reference, merely summarizing
each operation and the default kernel behavior when a NULL pointer is used.
PAGE 50
As you read through the list of file_operations methods, you will note that a number
of parameters include the string__user. This annotation is a form of documentation,
noting that a pointer is a user-space address that cannot be directly
dereferenced. For normal compilation, __user has no effect, but it can be used by
external checking software to find misuse of user-space addresses.
The rest of the chapter, after describing some other important data structures,
explains the role of the most important operations and offers hints, caveats, and real
code examples. We defer discussion of the more complex operations to later chapters,
because we aren't ready to dig into topics such as memory management, blocking
operations, and asynchronous notification quite yet.
struct module *owner
The first file_operations field is not an operation at all; it is a pointer to the
module that "owns" the structure. This field is used to prevent the module from
being unloaded while its operations are in use. Almost all the time, it is simply
initialized to THIS_MODULE, a macro defined in <linux/module.h>.
loff_t (*llseek) (struct file *, loff_t, int);
The llseek method is used to change the current read/write position in a file, and
the new position is returned as a (positive) return value. The loff_t parameter is
a "long offset" and is at least 64 bits wide even on 32-bit platforms. Errors are
signaled by a negative return value. If this function pointer is NULL, seek calls will
modify the position counter in the file structure (described in the section "The
file Structure") in potentially unpredictable ways.
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
Used to retrieve data from the device. A null pointer in this position causes the
read system call to fail with -EINVAL ("Invalid argument"). A nonnegative return
value represents the number of bytes successfully read (the return value is a
"signed size" type, usually the native integer type for the target platform).
ssize_t (*aio_read)(struct kiocb *, char __user *, size_t, loff_t);<br>
Initiates an asynchronous read--a read operation that might not complete
before the function returns. If this method is NULL, all operations will be processed
(synchronously) by read instead.
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
Sends data to the device. If NULL, -EINVAL is returned to the program calling the
write system call. The return value, if nonnegative, represents the number of
bytes successfully written.
ssize_t (*aio_write)(struct kiocb *, const char __user *, size_t, loff_t *);
Initiates an asynchronous write operation on the device.
int (*readdir) (struct file *, void *, filldir_t);
This field should be NULL for device files; it is used for reading directories and is
useful only for filesystems.
PAGE 51
unsigned int (*poll) (struct file *, struct poll_table_struct *);
The poll method is the back end of three system calls: poll, epoll, and select, all of
which are used to query whether a read or write to one or more file descriptors
would block. The poll method should return a bit mask indicating whether nonblocking
reads or writes are possible, and, possibly, provide the kernel with
information that can be used to put the calling process to sleep until I/O
becomes possible. If a driver leaves its poll method NULL, the device is assumed to
be both readable and writable without blocking.
int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
The ioctl system call offers a way to issue device-specific commands (such as formatting
a track of a floppy disk, which is neither reading nor writing). Additionally,
a few ioctl commands are recognized by the kernel without referring to the
fops table. If the device doesn't provide an ioctl method, the system call returns
an error for any request that isn't predefined (-ENOTTY, "No such ioctl for
device").
int (*mmap) (struct file *, struct vm_area_struct *);
mmap is used to request a mapping of device memory to a process's address
space. If this method is NULL, the mmap system call returns -ENODEV.
int (*open) (struct inode *, struct file *);
Though this is always the first operation performed on the device file, the driver
is not required to declare a corresponding method. If this entry is NULL, opening
the device always succeeds, but your driver isn't notified.
int (*flush) (struct file *);
The flush operation is invoked when a process closes its copy of a file descriptor
for a device; it should execute (and wait for) any outstanding operations on the
device. This must not be confused with the fsync operation requested by user
programs. Currently, flush is used in very few drivers; the SCSI tape driver uses
it, for example, to ensure that all data written makes it to the tape before the
device is closed. If flush is NULL, the kernel simply ignores the user application
request.
int (*release) (struct inode *, struct file *);
This operation is invoked when the file structure is being released. Like open,
release can be NULL.*
int (*fsync) (struct file *, struct dentry *, int);
This method is the back end of the fsync system call, which a user calls to flush
any pending data. If this pointer is NULL, the system call returns -EINVAL.
* Note that release isn't invoked every time a process calls close. Whenever
a file structure is shared (for example, after a fork or a dup), release won't
be invoked until all copies are closed. If you need to flush pending
data when any copy is closed, you should implement the flush method.
PAGE 52
int (*aio_fsync)(struct kiocb *, int);
This is the asynchronous version of the fsync method.
int (*fasync) (int, struct file *, int);
This operation is used to notify the device of a change in its FASYNC flag. Asynchronous
notification is an advanced topic and is described in Chapter 6. The
field can be NULL if the driver doesn't support asynchronous notification.
int (*lock) (struct file *, int, struct file_lock *);
The lock method is used to implement file locking; locking is an indispensable
feature for regular files but is almost never implemented by device drivers.
ssize_t (*readv) (struct file *, const struct iovec *, unsigned long, loff_t *);
ssize_t (*writev) (struct file *, const struct iovec *, unsigned long, loff_t *);
These methods implement scatter/gather read and write operations. Applications
occasionally need to do a single read or write operation involving multiple
memory areas; these system calls allow them to do so without forcing extra copy
operations on the data. If these function pointers are left NULL, the read and write
methods are called (perhaps more than once) instead.
ssize_t (*sendfile)(struct file *, loff_t *, size_t, read_actor_t, void *);
This method implements the read side of the sendfile system call, which moves
the data from one file descriptor to another with a minimum of copying. It is
used, for example, by a web server that needs to send the contents of a file out a
network connection. Device drivers usually leave sendfile NULL.
ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int);
sendpage is the other half of sendfile; it is called by the kernel to send data, one
page at a time, to the corresponding file. Device drivers do not usually implement
sendpage.
unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned
long, unsigned long, unsigned long);
The purpose of this method is to find a suitable location in the process's address
space to map in a memory segment on the underlying device. This task is normally
performed by the memory management code; this method exists to allow
drivers to enforce any alignment requirements a particular device may have.
Most drivers can leave this method NULL.
int (*check_flags)(int)
This method allows a module to check the flags passed to an fcntl(F_SETFL...) call.
int (*dir_notify)(struct file *, unsigned long);
This method is invoked when an application uses fcntl to request directory
change notifications. It is useful only to filesystems; drivers need not implement
dir_notify.
PAGE 53
The scull device driver implements only the most important device methods. Its
file_operations structure is initialized as follows:
struct file_operations scull_fops = {
.owner = THIS_MODULE,
.llseek = scull_llseek,
.read = scull_read,
.write = scull_write,
.ioctl = scull_ioctl,
.open = scull_open,
.release = scull_release,
};
This declaration uses the standard C tagged structure initialization syntax. This syntax
is preferred because it makes drivers more portable across changes in the definitions
of the structures and, arguably, makes the code more compact and readable.
Tagged initialization allows the reordering of structure members; in some cases, substantial
performance improvements have been realized by placing pointers to frequently
accessed members in the same hardware cache line.
The file Structure
struct file, defined in <linux/fs.h>, is the second most important data structure
used in device drivers. Note that a file has nothing to do with the FILE pointers of
user-space programs. A FILE is defined in the C library and never appears in kernel
code. A struct file, on the other hand, is a kernel structure that never appears in
user programs.
The file structure represents an open file. (It is not specific to device drivers; every
open file in the system has an associated struct file in kernel space.) It is created by
the kernel on open and is passed to any function that operates on the file, until
the last close. After all instances of the file are closed, the kernel releases the data
structure.
In the kernel sources, a pointer to struct file is usually called either file or filp
("file pointer"). We'll consistently call the pointer filp to prevent ambiguities with
the structure itself. Thus, file refers to the structure and filp to a pointer to the
structure.
The most important fields of struct file are shown here. As in the previous section,
the list can be skipped on a first reading. However, later in this chapter, when we
face some real C code, we'll discuss the fields in more detail.
mode_t f_mode;
The file mode identifies the file as either readable or writable (or both), by means
of the bits FMODE_READ and FMODE_WRITE. You might want to check this field for
read/write permission in your open or ioctl function, but you don't need to check
permissions for read and write, because the kernel checks before invoking your
PAGE 54
method. An attempt to read or write when the file has not been opened for that
type of access is rejected without the driver even knowing about it.
loff_t f_pos;
The current reading or writing position. loff_t is a 64-bit value on all platforms
(long long in gcc terminology). The driver can read this value if it needs to know
the current position in the file but should not normally change it; read and write
should update a position using the pointer they receive as the last argument
instead of acting on filp->f_pos directly. The one exception to this rule is in the
llseek method, the purpose of which is to change the file position.
unsigned int f_flags;
These are the file flags, such as O_RDONLY, O_NONBLOCK, and O_SYNC. A driver should
check the O_NONBLOCK flag to see if nonblocking operation has been requested (we
discuss nonblocking I/O in the section "Blocking and Nonblocking Operations"
in Chapter 6); the other flags are seldom used. In particular, read/write permission
should be checked using f_mode rather than f_flags. All the flags are
defined in the header <linux/fcntl.h>.
struct file_operations *f_op;
The operations associated with the file. The kernel assigns the pointer as part of
its implementation of open and then reads it when it needs to dispatch any operations.
The value in filp->f_op is never saved by the kernel for later reference;
this means that you can change the file operations associated with your file, and
the new methods will be effective after you return to the caller. For example, the
code for open associated with major number 1 (/dev/null, /dev/zero, and so on)
substitutes the operations in filp->f_op depending on the minor number being
opened. This practice allows the implementation of several behaviors under the
same major number without introducing overhead at each system call. The ability
to replace the file operations is the kernel equivalent of "method overriding"
in object-oriented programming.
void *private_data;
The open system call sets this pointer to NULL before calling the open method for
the driver. You are free to make its own use of the field or to ignore it; you can
use the field to point to allocated data, but then you must remember to free that
memory in the release method before the file structure is destroyed by the kernel.
private_data is a useful resource for preserving state information across system
calls and is used by most of our sample modules.
struct dentry *f_dentry;
The directory entry (dentry) structure associated with the file. Device driver writers
normally need not concern themselves with dentry structures, other than to
access the inode structure as filp->f_dentry->d_inode.
PAGE 55
The real structure has a few more fields, but they aren't useful to device drivers. We
can safely ignore those fields, because drivers never create file structures; they only
access structures created elsewhere.
The inode Structure
The inode structure is used by the kernel internally to represent files. Therefore, it is
different from the file structure that represents an open file descriptor. There can be
numerous file structures representing multiple open descriptors on a single file, but
they all point to a single inode structure.
The inode structure contains a great deal of information about the file. As a general
rule, only two fields of this structure are of interest for writing driver code:
dev_t i_rdev;
For inodes that represent device files, this field contains the actual device number.
struct cdev *i_cdev;
struct cdev is the kernel's internal structure that represents char devices; this
field contains a pointer to that structure when the inode refers to a char device
file.
The type of i_rdev changed over the course of the 2.5 development series, breaking a
lot of drivers. As a way of encouraging more portable programming, the kernel developers
have added two macros that can be used to obtain the major and minor number
from an inode:
unsigned int iminor(struct inode *inode);
unsigned int imajor(struct inode *inode);
In the interest of not being caught by the next change, these macros should be used
instead of manipulating i_rdev directly.
Char Device Registration
As we mentioned, the kernel uses structures of type struct cdev to represent char
devices internally. Before the kernel invokes your device's operations, you must allocate
and register one or more of these structures.* To do so, your code should include
<linux/cdev.h>, where the structure and its associated helper functions are defined.
There are two ways of allocating and initializing one of these structures. If you wish
to obtain a stand alone cdev structure at runtime, you may do so with code such as:
struct cdev *my_cdev = cdev_alloc( );
my_cdev->ops = &my_fops;
* There is an older mechanism that avoids the use of cdev structures (which we discuss
in the section "The Older Way"). New code should use the newer technique, however.
PAGE 56
Chances are, however, that you will want to embed the cdev structure within a
device-specific structure of your own; that is what scull does. In that case, you should
initialize the structure that you have already allocated with:
void cdev_init(struct cdev *cdev, struct file_operations *fops);
Either way, there is one other struct cdev field that you need to initialize. Like the
file_operations structure, struct cdev has an owner field that should be set to
THIS_MODULE.
Once the cdev structure is set up, the final step is to tell the kernel about it with a call to:
int cdev_add(struct cdev *dev, dev_t num, unsigned int count);
Here, dev is the cdev structure, num is the first device number to which this device
responds, and count is the number of device numbers that should be associated with
the device. Often count is one, but there are situations where it makes sense to have
more than one device number correspond to a specific device. Consider, for example,
the SCSI tape driver, which allows user space to select operating modes (such as
density) by assigning multiple minor numbers to each physical device.
There are a couple of important things to keep in mind when using cdev_add. The
first is that this call can fail. If it returns a negative error code, your device has not
been added to the system. It almost always succeeds, however, and that brings up
the other point: as soon as cdev_add returns, your device is "live" and its operations
can be called by the kernel. You should not call cdev_add until your driver is completely
ready to handle operations on the device.
To remove a char device from the system, call:
void cdev_del(struct cdev *dev);
Clearly, you should not access the cdev structure after passing it to cdev_del.
Device Registration in scull
Internally, scull represents each device with a structure of type struct scull_dev. This
structure is defined as:
struct scull_dev {
struct scull_qset *data; /* Pointer to first quantum set */
int quantum; /* the current quantum size */
int qset; /* the current array size */
unsigned long size; /* amount of data stored here */
unsigned int access_key; /* used by sculluid and scullpriv */
struct semaphore sem; /* mutual exclusion semaphore */
struct cdev cdev; /* Char device structure */
};
We discuss the various fields in this structure as we come to them, but for now, we
call attention to cdev, the struct cdev that interfaces our device to the kernel. This
PAGE 57
structure must be initialized and added to the system as described above; the scull
code that handles this task is:
static void scull_setup_cdev(struct scull_dev *dev, int index)
{
int err, devno = MKDEV(scull_major, scull_minor + index);
cdev_init(&dev->cdev, &scull_fops);
dev->cdev.owner = THIS_MODULE;
dev->cdev.ops = &scull_fops;
err = cdev_add (&dev->cdev, devno, 1);
/* Fail gracefully if need be */
if (err)
printk(KERN_NOTICE "Error %d adding scull%d", err, index);
}
Since the cdev structure is embedded within struct scull_dev, cdev_init must be
called to perform the initialization of that structure.
The Older Way
If you dig through much driver code in the 2.6 kernel, you may notice that quite a
few char drivers do not use the cdev interface that we have just described. What you
are seeing is older code that has not yet been upgraded to the 2.6 interface. Since that
code works as it is, this upgrade may not happen for a long time. For completeness,
we describe the older char device registration interface, but new code should not use
it; this mechanism will likely go away in a future kernel.
The classic way to register a char device driver is with:
int register_chrdev(unsigned int major, const char *name,
struct file_operations *fops);
Here, major is the major number of interest, name is the name of the driver (it
appears in /proc/devices), and fops is the default file_operations structure. A call to
register_chrdev registers minor numbers 0-255 for the given major, and sets up a
default cdev structure for each. Drivers using this interface must be prepared to handle
open calls on all 256 minor numbers (whether they correspond to real devices or
not), and they cannot use major or minor numbers greater than 255.
If you use register_chrdev, the proper function to remove your device(s) from the system
is:
int unregister_chrdev(unsigned int major, const char *name);
major and name must be the same as those passed to register_chrdev, or the call will
fail.
PAGE 58
open and release
Now that we've taken a quick look at the fields, we start using them in real scull
functions.
The open Method
The open method is provided for a driver to do any initialization in preparation for
later operations. In most drivers, open should perform the following tasks:
- Check for device-specific errors (such as device-not-ready or similar hardware problems)
- Initialize the device if it is being opened for the first time
- Update the f_op pointer, if necessary
- Allocate and fill any data structure to be put in filp->private_data
The first order of business, however, is usually to identify which device is being
opened. Remember that the prototype for the open method is:
int (*open)(struct inode *inode, struct file *filp);
The inode argument has the information we need in the form of its i_cdev field,
which contains the cdev structure we set up before. The only problem is that we do
not normally want the cdev structure itself, we want the scull_dev structure that contains
that cdev structure. The C language lets programmers play all sorts of tricks to
make that kind of conversion; programming such tricks is error prone, however, and
leads to code that is difficult for others to read and understand. Fortunately, in this
case, the kernel hackers have done the tricky stuff for us, in the form of the
container_of macro, defined in <linux/kernel.h>:
container_of(pointer, container_type, container_field);
This macro takes a pointer to a field named container_field, within a structure of
type container_type, and returns a pointer to the containing structure. In scull_open,
this macro is used to find the appropriate device structure:
struct scull_dev *dev; /* device information */
dev = container_of(inode->i_cdev, struct scull_dev, cdev);
filp->private_data = dev; /* for other methods */
Once it has found the scull_dev structure, scull stores a pointer to it in the private_data
field of the file structure for easier access in the future.
The other way to identify the device being opened is to look at the minor number
stored in the inode structure. If you register your device with register_chrdev, you
must use this technique. Be sure to use iminor to obtain the minor number from the
inode structure, and make sure that it corresponds to a device that your driver is
actually prepared to handle.
PAGE 59
The (slightly simplified) code for scull_open is:
int scull_open(struct inode *inode, struct file *filp)
{
struct scull_dev *dev; /* device information */
dev = container_of(inode->i_cdev, struct scull_dev, cdev);
filp->private_data = dev; /* for other methods */
/* now trim to 0 the length of the device if open was write-only */
if ( (filp->f_flags & O_ACCMODE) = = O_WRONLY) {
scull_trim(dev); /* ignore errors */
}
return 0; /* success */
}
The code looks pretty sparse, because it doesn't do any particular device handling
when open is called. It doesn't need to, because the scull device is global and persistent
by design. Specifically, there's no action such as "initializing the device on first
open," because we don't keep an open count for sculls.
The only real operation performed on the device is truncating it to a length of 0 when
the device is opened for writing. This is performed because, by design, overwriting a
scull device with a shorter file results in a shorter device data area. This is similar to
the way opening a regular file for writing truncates it to zero length. The operation
does nothing if the device is opened for reading.
We'll see later how a real initialization works when we look at the code for the other
scull personalities.
The release Method
The role of the release method is the reverse of open. Sometimes you'll find that the
method implementation is called device_close instead of device_release. Either
way, the device method should perform the following tasks:
- Deallocate anything that open allocated in filp->private_data
- Shut down the device on last close
The basic form of scull has no hardware to shut down, so the code required is
minimal:*
int scull_release(struct inode *inode, struct file *filp)
{
return 0;
}
* The other flavors of the device are closed by different functions because scull_open substituted a different
filp->f_op for each device. We'll discuss these as we introduce each flavor.
PAGE 60
You may be wondering what happens when a device file is closed more times than it
is opened. After all, the dup and fork system calls create copies of open files without
calling open; each of those copies is then closed at program termination. For example,
most programs don't open their stdin file (or device), but all of them end up closing
it. How does a driver know when an open device file has really been closed?
The answer is simple: not every close system call causes the release method to be
invoked. Only the calls that actually release the device data structure invoke the
method--hence its name. The kernel keeps a counter of how many times a file
structure is being used. Neither fork nor dup creates a new file structure (only open
does that); they just increment the counter in the existing structure. The close system
call executes the release method only when the counter for the file structure
drops to 0, which happens when the structure is destroyed. This relationship
between the release method and the close system call guarantees that your driver sees
only one release call for each open.
Note that the flush method is called every time an application calls close. However,
very few drivers implement flush, because usually there's nothing to perform at close
time unless release is involved.
As you may imagine, the previous discussion applies even when the application terminates
without explicitly closing its open files: the kernel automatically closes any
file at process exit time by internally using the close system call.
scull's Memory Usage
Before introducing the read and write operations, we'd better look at how and why
scull performs memory allocation. "How" is needed to thoroughly understand the
code, and "why" demonstrates the kind of choices a driver writer needs to make,
although scull is definitely not typical as a device.
This section deals only with the memory allocation policy in scull and doesn't show
the hardware management skills you need to write real drivers. These skills are introduced
in Chapters 9 and 10. Therefore, you can skip this section if you're not interested
in understanding the inner workings of the memory-oriented scull driver.
The region of memory used by scull, also called a device, is variable in length. The
more you write, the more it grows; trimming is performed by overwriting the device
with a shorter file.
The scull driver introduces two core functions used to manage memory in the Linux
kernel. These functions, defined in <linux/slab.h>, are:
void *kmalloc(size_t size, int flags);
void kfree(void *ptr);
A call to kmalloc attempts to allocate size bytes of memory; the return value is a
pointer to that memory or NULL if the allocation fails. The flags argument is used to
PAGE 61
describe how the memory should be allocated; we examine those flags in detail in
Chapter 8. For now, we always use GFP_KERNEL. Allocated memory should be freed
with kfree. You should never pass anything to kfree that was not obtained from
kmalloc. It is, however, legal to pass a NULL pointer to kfree.
kmalloc is not the most efficient way to allocate large areas of memory (see
Chapter 8), so the implementation chosen for scull is not a particularly smart one.
The source code for a smart implementation would be more difficult to read, and the
aim of this section is to show read and write, not memory management. That's why
the code just uses kmalloc and kfree without resorting to allocation of whole pages,
although that approach would be more efficient.
On the flip side, we didn't want to limit the size of the "device" area, for both a
philosophical reason and a practical one. Philosophically, it's always a bad idea to
put arbitrary limits on data items being managed. Practically, scull can be used to
temporarily eat up your system's memory in order to run tests under low-memory
conditions. Running such tests might help you understand the system's internals.
You can use the command cp /dev/zero /dev/scull0 to eat all the real RAM with scull,
and you can use the dd utility to choose how much data is copied to the scull device.
In scull, each device is a linked list of pointers, each of which points to a scull_qset
structure. Each such structure can refer, by default, to at most four million bytes,
through an array of intermediate pointers. The released source uses an array of 1000
pointers to areas of 4000 bytes. We call each memory area a quantum and the array
(or its length) a quantum set. A scull device and its memory areas are shown in
Figure 3-1.
Figure 3-1. The layout of a scull device
PAGE 62
The chosen numbers are such that writing a single byte in scull consumes 8000 or
12,000 bytes of memory: 4000 for the quantum and 4000 or 8000 for the
quantum set (according to whether a pointer is represented in 32 bits or 64 bits on
the target platform). If, instead, you write a huge amount of data, the overhead of the
linked list is not too bad. There is only one list element for every four megabytes of
data, and the maximum size of the device is limited by the computer's memory size.
Choosing the appropriate values for the quantum and the quantum set is a question
of policy, rather than mechanism, and the optimal sizes depend on how the device is
used. Thus, the scull driver should not force the use of any particular values for the
quantum and quantum set sizes. In scull, the user can change the values in
several ways: by changing the macros SCULL_QUANTUM and SCULL_QSET in scull.h at
compile time, by setting the integer values scull_quantum and scull_qset at module
load time, or by changing both the current and default values using ioctl at runtime.
Using a macro and an integer value to allow both compile-time and load-time configuration
is reminiscent of how the major number is selected. We use this technique
for whatever value in the driver is arbitrary or related to policy.
The only question left is how the default numbers have been chosen. In this particular
case, the problem is finding the best balance between the waste of memory resulting
from half-filled quanta and quantum sets and the overhead of allocation,
deallocation, and pointer chaining that occurs if quanta and sets are small. Additionally,
the internal design of kmalloc should be taken into account. (We won't pursue
the point now, though; the innards of kmalloc are explored in Chapter 8.) The choice
of default numbers comes from the assumption that massive amounts of data are
likely to be written to scull while testing it, although normal use of the device will
most likely transfer just a few kilobytes of data.
We have already seen the scull_dev structure that represents our device internally.
That structure's quantum and qset fields hold the device's quantum and quantum set
sizes, respectively. The actual data, however, is tracked by a different structure,
which we call struct scull_qset:
struct scull_qset {
void **data;
struct scull_qset *next;
};
The next code fragment shows in practice how struct scull_dev and struct scull_qset
are used to hold data. The function scull_trim is in charge of freeing the whole data
area and is invoked by scull_open when the file is opened for writing. It simply walks
through the list and frees any quantum and quantum set it finds.
int scull_trim(struct scull_dev *dev)
{
struct scull_qset *next, *dptr;
int qset = dev->qset; /* "dev" is not-null */
int i;
PAGE 63
for (dptr = dev->data; dptr; dptr = next) { /* all the list items */
if (dptr->data) {
for (i = 0; i < qset; i++)
kfree(dptr->data[i]);
kfree(dptr->data);
dptr->data = NULL;
}
next = dptr->next;
kfree(dptr);
}
dev->size = 0;
dev->quantum = scull_quantum;
dev->qset = scull_qset;
dev->data = NULL;
return 0;
}
scull_trim is also used in the module cleanup function to return memory used by
scull to the system.
read and write
The read and write methods both perform a similar task, that is, copying data from
and to application code. Therefore, their prototypes are pretty similar, and it's worth
introducing them at the same time:
ssize_t read(struct file *filp, char __user *buff,
size_t count, loff_t *offp);
ssize_t write(struct file *filp, const char __user *buff,
size_t count, loff_t *offp);
For both methods, filp is the file pointer and count is the size of the requested data
transfer. The buff argument points to the user buffer holding the data to be written or
the empty buffer where the newly read data should be placed. Finally, offp is a pointer
to a "long offset type" object that indicates the file position the user is accessing. The
return value is a "signed size type"; its use is discussed later.
Let us repeat that the buff argument to the read and write methods is a user-space
pointer. Therefore, it cannot be directly dereferenced by kernel code. There are a few
reasons for this restriction:
- Depending on which architecture your driver is running on, and how the kernel
was configured, the user-space pointer may not be valid while running in kernel
mode at all. There may be no mapping for that address, or it could point to some
other, random data.
- Even if the pointer does mean the same thing in kernel space, user-space memory
is paged, and the memory in question might not be resident in RAM when
the system call is made. Attempting to reference the user-space memory directly
could generate a page fault, which is something that kernel code is not allowed
PAGE 64
to do. The result would be an "oops," which would result in the death of the
process that made the system call.
- The pointer in question has been supplied by a user program, which could be
buggy or malicious. If your driver ever blindly dereferences a user-supplied
pointer, it provides an open doorway allowing a user-space program to access or
overwrite memory anywhere in the system. If you do not wish to be responsible
for compromising the security of your users' systems, you cannot ever dereference
a user-space pointer directly.
Obviously, your driver must be able to access the user-space buffer in order to get its
job done. This access must always be performed by special, kernel-supplied functions,
however, in order to be safe. We introduce some of those functions (which are
defined in <asm/uaccess.h>) here, and the rest in the section "Using the ioctl Argument"
in Chapter 6; they use some special, architecture-dependent magic to ensure
that data transfers between kernel and user space happen in a safe and correct way.
The code for read and write in scull needs to copy a whole segment of data to or from
the user address space. This capability is offered by the following kernel functions,
which copy an arbitrary array of bytes and sit at the heart of most read and write
implementations:
unsigned long copy_to_user(void __user *to,
const void *from,
unsigned long count);
unsigned long copy_from_user(void *to,
const void __user *from,
unsigned long count);
Although these functions behave like normal memcpy functions, a little extra care
must be used when accessing user space from kernel code. The user pages being
addressed might not be currently present in memory, and the virtual memory subsystem
can put the process to sleep while the page is being transferred into place.
This happens, for example, when the page must be retrieved from swap space. The
net result for the driver writer is that any function that accesses user space must be
reentrant, must be able to execute concurrently with other driver functions, and, in
particular, must be in a position where it can legally sleep. We return to this subject
in Chapter 5.
The role of the two functions is not limited to copying data to and from user-space:
they also check whether the user space pointer is valid. If the pointer is invalid, no copy
is performed; if an invalid address is encountered during the copy, on the other hand,
only part of the data is copied. In both cases, the return value is the amount of memory
still to be copied. The scull code looks for this error return, and returns -EFAULT to
the user if it's not 0.
The topic of user-space access and invalid user space pointers is somewhat advanced
and is discussed in Chapter 6. However, it's worth noting that if you don't need to
PAGE 65
check the user-space pointer you can invoke __copy_to_user and __copy_from_user
instead. This is useful, for example, if you know you already checked the argument.
Be careful, however; if, in fact, you do not check a user-space pointer that you pass to
these functions, then you can create kernel crashes and/or security holes.
As far as the actual device methods are concerned, the task of the read method is to
copy data from the device to user space (using copy_to_user), while the write method
must copy data from user space to the device (using copy_from_user). Each read or
write system call requests transfer of a specific number of bytes, but the driver is free
to transfer less data--the exact rules are slightly different for reading and writing and
are described later in this chapter.
Whatever the amount of data the methods transfer, they should generally update the
file position at *offp to represent the current file position after successful completion
of the system call. The kernel then propagates the file position change back into
the file structure when appropriate. The pread and pwrite system calls have different
semantics, however; they operate from a given file offset and do not change the
file position as seen by any other system calls. These calls pass in a pointer to the
user-supplied position, and discard the changes that your driver makes.
Figure 3-2. The arguments to read
Both the read and write methods return a negative value if an error occurs. A return
value greater than or equal to 0, instead, tells the calling program how many bytes
have been successfully transferred. If some data is transferred correctly and then an
error happens, the return value must be the count of bytes successfully transferred,
PAGE 66
and the error does not get reported until the next time the function is called. Implementing
this convention requires, of course, that your driver remember that the error
has occurred so that it can return the error status in the future.
Although kernel functions return a negative number to signal an error, and the value
of the number indicates the kind of error that occurred (as introduced in Chapter 2),
programs that run in user space always see -1 as the error return value. They need to
access the errno variable to find out what happened. The user-space behavior is dictated
by the POSIX standard, but that standard does not make requirements on how
the kernel operates internally.
The read Method
The return value for read is interpreted by the calling application program:
- If the value equals the count argument passed to the read system call, the
requested number of bytes has been transferred. This is the optimal case.
- If the value is positive, but smaller than count, only part of the data has been
transferred. This may happen for a number of reasons, depending on the device.
Most often, the application program retries the read. For instance, if you read
using the fread function, the library function reissues the system call until completion
of the requested data transfer.
- If the value is 0, end-of-file was reached (and no data was read).
- A negative value means there was an error. The value specifies what the error
was, according to <linux/errno.h>. Typical values returned on error include -EINTR
(interrupted system call) or -EFAULT (bad address).
What is missing from the preceding list is the case of "there is no data, but it may
arrive later." In this case, the read system call should block. We'll deal with blocking
input in Chapter 6.
The scull code takes advantage of these rules. In particular, it takes advantage of the
partial-read rule. Each invocation of scull_read deals only with a single data quantum,
without implementing a loop to gather all the data; this makes the code shorter
and easier to read. If the reading program really wants more data, it reiterates the
call. If the standard I/O library (i.e., fread) is used to read the device, the application
won't even notice the quantization of the data transfer.
If the current read position is greater than the device size, the read method of scull
returns 0 to signal that there's no data available (in other words, we're at end-of-file).
This situation can happen if process A is reading the device while process B opens it
for writing, thus truncating the device to a length of 0. Process A suddenly finds itself
past end-of-file, and the next read call returns 0.
PAGE 67
Here is the code for read (ignore the calls to down_interruptible and up for now; we
will get to them in the next chapter):
ssize_t scull_read(struct file *filp, char __user *buf, size_t count,
loff_t *f_pos)
{
struct scull_dev *dev = filp->private_data;
struct scull_qset *dptr; /* the first listitem */
int quantum = dev->quantum, qset = dev->qset;
int itemsize = quantum * qset; /* how many bytes in the listitem */
int item, s_pos, q_pos, rest;
ssize_t retval = 0;
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
if (*f_pos >= dev->size)
goto out;
if (*f_pos + count > dev->size)
count = dev->size - *f_pos;
/* find listitem, qset index, and offset in the quantum */
item = (long)*f_pos / itemsize;
rest = (long)*f_pos % itemsize;
s_pos = rest / quantum; q_pos = rest % quantum;
/* follow the list up to the right position (defined elsewhere) */
dptr = scull_follow(dev, item);
if (dptr = = NULL || !dptr->data || ! dptr->data[s_pos])
goto out; /* don't fill holes */
/* read only up to the end of this quantum */
if (count > quantum - q_pos)
count = quantum - q_pos;
if (copy_to_user(buf, dptr->data[s_pos] + q_pos, count)) {
retval = -EFAULT;
goto out;
}
*f_pos += count;
retval = count;
out:
up(&dev->sem);
return retval;
}
PAGE 68
The write Method
write, like read, can transfer less data than was requested, according to the following
rules for the return value:
- If the value equals count, the requested number of bytes has been transferred.
- If the value is positive, but smaller than count, only part of the data has been
transferred. The program will most likely retry writing the rest of the data.
- If the value is 0, nothing was written. This result is not an error, and there is no
reason to return an error code. Once again, the standard library retries the call to
write. We'll examine the exact meaning of this case in Chapter 6, where blocking
write is introduced.
- A negative value means an error occurred; as for read, valid error values are
those defined in <linux/errno.h>.
Unfortunately, there may still be misbehaving programs that issue an error message
and abort when a partial transfer is performed. This happens because some programmers
are accustomed to seeing write calls that either fail or succeed completely,
which is actually what happens most of the time and should be supported by devices
as well. This limitation in the scull implementation could be fixed, but we didn't
want to complicate the code more than necessary.
The scull code for write deals with a single quantum at a time, as the read method
does:
ssize_t scull_write(struct file *filp, const char __user *buf, size_t count,
loff_t *f_pos)
{
struct scull_dev *dev = filp->private_data;
struct scull_qset *dptr;
int quantum = dev->quantum, qset = dev->qset;
int itemsize = quantum * qset;
int item, s_pos, q_pos, rest;
ssize_t retval = -ENOMEM; /* value used in "goto out" statements */
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
/* find listitem, qset index and offset in the quantum */
item = (long)*f_pos / itemsize;
rest = (long)*f_pos % itemsize;
s_pos = rest / quantum; q_pos = rest % quantum;
/* follow the list up to the right position */
dptr = scull_follow(dev, item);
if (dptr = = NULL)
goto out;
if (!dptr->data) {
dptr->data = kmalloc(qset * sizeof(char *), GFP_KERNEL);
if (!dptr->data)
PAGE 69
goto out;
memset(dptr->data, 0, qset * sizeof(char *));
}
if (!dptr->data[s_pos]) {
dptr->data[s_pos] = kmalloc(quantum, GFP_KERNEL);
if (!dptr->data[s_pos])
goto out;
}
/* write only up to the end of this quantum */
if (count > quantum - q_pos)
count = quantum - q_pos;
if (copy_from_user(dptr->data[s_pos]+q_pos, buf, count)) {
retval = -EFAULT;
goto out;
}
*f_pos += count;
retval = count;
/* update the size */
if (dev->size < *f_pos)
dev->size = *f_pos;
out:
up(&dev->sem);
return retval;
}
readv and writev
Unix systems have long supported two system calls named readv and writev. These
"vector" versions of read and write take an array of structures, each of which contains
a pointer to a buffer and a length value. A readv call would then be expected to
read the indicated amount into each buffer in turn. writev, instead, would gather
together the contents of each buffer and put them out as a single write operation.
If your driver does not supply methods to handle the vector operations, readv and
writev are implemented with multiple calls to your read and write methods. In many
situations, however, greater efficiency is achieved by implementing readv and writev
directly.
The prototypes for the vector operations are:
ssize_t (*readv) (struct file *filp, const struct iovec *iov,
unsigned long count, loff_t *ppos);
ssize_t (*writev) (struct file *filp, const struct iovec *iov,
unsigned long count, loff_t *ppos);
Here, the filp and ppos arguments are the same as for read and write.
The iovec structure, defined in <linux/uio.h>, looks like:
struct iovec
{
PAGE 70
void __user *iov_base;
__kernel_size_t iov_len;
};
Each iovec describes one chunk of data to be transferred; it starts at iov_base (in user
space) and is iov_len bytes long. The count parameter tells the method how many
iovec structures there are. These structures are created by the application, but the
kernel copies them into kernel space before calling the driver.
The simplest implementation of the vectored operations would be a straightforward
loop that just passes the address and length out of each iovec to the driver's read or
write function. Often, however, efficient and correct behavior requires that the driver
do something smarter. For example, a writev on a tape drive should write the contents
of all the iovec structures as a single record on the tape.
Many drivers, however, gain no benefit from implementing these methods themselves.
Therefore, scull omits them. The kernel emulates them with read and write,
and the end result is the same.
Playing with the New Devices
Once you are equipped with the four methods just described, the driver can be compiled
and tested; it retains any data you write to it until you overwrite it with new
data. The device acts like a data buffer whose length is limited only by the amount of
real RAM available. You can try using cp, dd, and input/output redirection to test out
the driver.
The free command can be used to see how the amount of free memory shrinks and
expands according to how much data is written into scull.
To get more confident with reading and writing one quantum at a time, you can add
a printk at an appropriate point in the driver and watch what happens while an application
reads or writes large chunks of data. Alternatively, use the strace utility to
monitor the system calls issued by a program, together with their return values. Tracing
a cp or an ls -l > /dev/scull0 shows quantized reads and writes. Monitoring (and
debugging) techniques are presented in detail in Chapter 4
Quick Reference
This chapter introduced the following symbols and header files. The list of the fields
in struct file_operations and struct file is not repeated here.
PAGE 71
#include <linux/types.h>
dev_t
dev_t is the type used to represent device numbers within the kernel.
int MAJOR(dev_t dev);
int MINOR(dev_t dev);
Macros that extract the major and minor numbers from a device number.
dev_t MKDEV(unsigned int major, unsigned int minor);
Macro that builds a dev_t data item from the major and minor numbers.
#include <linux/fs.h>
The "filesystem" header is the header required for writing device drivers. Many
important functions and data structures are declared in here.
int register_chrdev_region(dev_t first, unsigned int count, char *name)
int alloc_chrdev_region(dev_t *dev, unsigned int firstminor, unsigned int count, char *name)
void unregister_chrdev_region(dev_t first, unsigned int count);
Functions that allow a driver to allocate and free ranges of device numbers.
register_chrdev_region should be used when the desired major number is known
in advance; for dynamic allocation, use alloc_chrdev_region instead.
int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);
The old (pre-2.6) char device registration routine. It is emulated in the 2.6 kernel
but should not be used for new code. If the major number is not 0, it is used
unchanged; otherwise a dynamic number is assigned for this device.
int unregister_chrdev(unsigned int major, const char *name);
Function that undoes a registration made with register_chrdev. Both major and
the name string must contain the same values that were used to register the driver.
struct file_operations;
struct file;
struct inode;
Three important data structures used by most device drivers. The file_operations
structure holds a char driver's methods; struct file represents an open file, and
struct inode represents a file on disk.
#include <linux/cdev.h>
struct cdev *cdev_alloc(void);
void cdev_init(struct cdev *dev, struct file_operations *fops);
int cdev_add(struct cdev *dev, dev_t num, unsigned int count);
void cdev_del(struct cdev *dev);
Functions for the management of cdev structures, which represent char devices
within the kernel.
PAGE 72
#include <linux/kernel.h>
container_of(pointer, type, field);
A convenience macro that may be used to obtain a pointer to a structure from a
pointer to some other structure contained within it.
#include <asm/uaccess.h>
This include file declares functions used by kernel code to move data to and
from user space.
unsigned long copy_from_user (void *to, const void *from, unsigned long count);
unsigned long copy_to_user (void *to, const void *from, unsigned long count);
Copy data between user space and kernel space.
PAGE 73
CHAPTER 4
Debugging Techniques
Kernel programming brings its own, unique debugging challenges. Kernel code cannot
be easily executed under a debugger, nor can it be easily traced, because it is a set
of functionalities not related to a specific process. Kernel code errors can also be
exceedingly hard to reproduce and can bring down the entire system with them, thus
destroying much of the evidence that could be used to track them down.
This chapter introduces techniques you can use to monitor kernel code and trace
errors under such trying circumstances.
Debugging Support in the Kernel
In Chapter 2, we recommended that you build and install your own kernel, rather
than running the stock kernel that comes with your distribution. One of the strongest
reasons for running your own kernel is that the kernel developers have built several
debugging features into the kernel itself. These features can create extra output
and slow performance, so they tend not to be enabled in production kernels from
distributors. As a kernel developer, however, you have different priorities and will
gladly accept the (minimal) overhead of the extra kernel debugging support.
Here, we list the configuration options that should be enabled for kernels used for
development. Except where specified otherwise, all of these options are found under
the "kernel hacking" menu in whatever kernel configuration tool you prefer. Note
that some of these options are not supported by all architectures.
CONFIG_DEBUG_KERNEL
This option just makes other debugging options available; it should be turned on
but does not, by itself, enable any features.
CONFIG_DEBUG_SLAB
This crucial option turns on several types of checks in the kernel memory allocation
functions; with these checks enabled, it is possible to detect a number of
memory overrun and missing initialization errors. Each byte of allocated memory
PAGE 74
is set to 0xa5 before being handed to the caller and then set to 0x6b when it is
freed. If you ever see either of those "poison" patterns repeating in output from
your driver (or often in an oops listing), you'll know exactly what sort of error to
look for. When debugging is enabled, the kernel also places special guard values
before and after every allocated memory object; if those values ever get changed,
the kernel knows that somebody has overrun a memory allocation, and it complains
loudly. Various checks for more obscure errors are enabled as well.
CONFIG_DEBUG_PAGEALLOC
Full pages are removed from the kernel address space when freed. This option
can slow things down significantly, but it can also quickly point out certain
kinds of memory corruption errors.
CONFIG_DEBUG_SPINLOCK
With this option enabled, the kernel catches operations on uninitialized spinlocks
and various other errors (such as unlocking a lock twice).
CONFIG_DEBUG_SPINLOCK_SLEEP
This option enables a check for attempts to sleep while holding a spinlock. In
fact, it complains if you call a function that could potentially sleep, even if the
call in question would not sleep.
CONFIG_INIT_DEBUG
Items marked with __init (or __initdata) are discarded after system initialization
or module load time. This option enables checks for code that attempts to
access initialization-time memory after initialization is complete.
CONFIG_DEBUG_INFO
This option causes the kernel to be built with full debugging information
included. You'll need that information if you want to debug the kernel with gdb.
You may also want to enable CONFIG_FRAME_POINTER if you plan to use gdb.
CONFIG_MAGIC_SYSRQ
Enables the "magic SysRq" key. We look at this key in the section "System
Hangs," later in this chapter.
CONFIG_DEBUG_STACKOVERFLOW
CONFIG_DEBUG_STACK_USAGE
These options can help track down kernel stack overflows. A sure sign of a stack
overflow is an oops listing without any sort of reasonable back trace. The first
option adds explicit overflow checks to the kernel; the second causes the kernel
to monitor stack usage and make some statistics available via the magic SysRq
key.
CONFIG_KALLSYMS
This option (under "General setup/Standard features") causes kernel symbol
information to be built into the kernel; it is enabled by default. The symbol
information is used in debugging contexts; without it, an oops listing can give
you a kernel traceback only in hexadecimal, which is not very useful.
PAGE 75
CONFIG_IKCONFIG
CONFIG_IKCONFIG_PROC
These options (found in the "General setup" menu) cause the full kernel configuration
state to be built into the kernel and to be made available via /proc. Most
kernel developers know which configuration they used and do not need these
options (which make the kernel bigger). They can be useful, though, if you are
trying to debug a problem in a kernel built by somebody else.
CONFIG_ACPI_DEBUG
Under "Power management/ACPI." This option turns on verbose ACPI
(Advanced Configuration and Power Interface) debugging information, which
can be useful if you suspect a problem related to ACPI.
CONFIG_DEBUG_DRIVER
Under "Device drivers." Turns on debugging information in the driver core,
which can be useful for tracking down problems in the low-level support code.
We'll look at the driver core in Chapter 14.
CONFIG_SCSI_CONSTANTS
This option, found under "Device drivers/SCSI device support," builds in information
for verbose SCSI error messages. If you are working on a SCSI driver, you
probably want this option.
CONFIG_INPUT_EVBUG
This option (under "Device drivers/Input device support") turns on verbose logging
of input events. If you are working on a driver for an input device, this
option may be helpful. Be aware of the security implications of this option, however:
it logs everything you type, including your passwords.
CONFIG_PROFILING
This option is found under "Profiling support." Profiling is normally used for
system performance tuning, but it can also be useful for tracking down some
kernel hangs and related problems.
We will revisit some of the above options as we look at various ways of tracking
down kernel problems. But first, we will look at the classic debugging technique:
print statements.
Debugging by Printing
The most common debugging technique is monitoring, which in applications programming
is done by calling printf at suitable points. When you are debugging kernel
code, you can accomplish the same goal with printk.
PAGE 76
printk
We used the printk function in earlier chapters with the simplifying assumption that
it works like printf. Now it's time to introduce some of the differences.
One of the differences is that printk lets you classify messages according to their
severity by associating different loglevels, or priorities, with the messages. You usually
indicate the loglevel with a macro. For example, KERN_INFO, which we saw
prepended to some of the earlier print statements, is one of the possible loglevels of
the message. The loglevel macro expands to a string, which is concatenated to the
message text at compile time; that's why there is no comma between the priority and
the format string in the following examples. Here are two examples of printk commands,
a debug message and a critical message:
printk(KERN_DEBUG "Here I am: %s:%i\n", __FILE__, __LINE__);
printk(KERN_CRIT "I'm trashed; giving up on %p\n", ptr);
There are eight possible loglevel strings, defined in the header <linux/kernel.h>; we
list them in order of decreasing severity:
KERN_EMERG
Used for emergency messages, usually those that precede a crash.
KERN_ALERT
A situation requiring immediate action.
KERN_CRIT
Critical conditions, often related to serious hardware or software failures.
KERN_ERR
Used to report error conditions; device drivers often use KERN_ERR to report hardware
difficulties.
KERN_WARNING
Warnings about problematic situations that do not, in themselves, create serious
problems with the system.
KERN_NOTICE
Situations that are normal, but still worthy of note. A number of security-related
conditions are reported at this level.
KERN_INFO
Informational messages. Many drivers print information about the hardware
they find at startup time at this level.
KERN_DEBUG
Used for debugging messages.
Each string (in the macro expansion) represents an integer in angle brackets. Integers
range from 0 to 7, with smaller values representing higher priorities.
PAGE 77
A printk statement with no specified priority defaults to DEFAULT_MESSAGE_LOGLEVEL,
specified in kernel/printk.c as an integer. In the 2.6.10 kernel, DEFAULT_MESSAGE_LOGLEVEL
is KERN_WARNING, but that has been known to change in the past.
Based on the loglevel, the kernel may print the message to the current console, be it a
text-mode terminal, a serial port, or a parallel printer. If the priority is less than the
integer variable console_loglevel, the message is delivered to the console one line at
a time (nothing is sent unless a trailing newline is provided). If both klogd and syslogd
are running on the system, kernel messages are appended to /var/log/messages
(or otherwise treated depending on your syslogd configuration), independent of
console_loglevel. If klogd is not running, the message won't reach user space unless
you read /proc/kmsg (which is often most easily done with the dmesg command).
When using klogd, you should remember that it doesn't save consecutive identical
lines; it only saves the first such line and, at a later time, the number of repetitions it
received.
The variable console_loglevel is initialized to DEFAULT_CONSOLE_LOGLEVEL and can be
modified through the sys_syslog system call. One way to change it is by specifying
the -c switch when invoking klogd, as specified in the klogd manpage. Note that to
change the current value, you must first kill klogd and then restart it with the -c
option. Alternatively, you can write a program to change the console loglevel. You'll
find a version of such a program in misc-progs/setlevel.c in the source files provided
on O'Reilly's FTP site. The new level is specified as an integer value between 1 and 8,
inclusive. If it is set to 1, only messages of level 0 (KERN_EMERG) reach the console; if it
is set to 8, all messages, including debugging ones, are displayed.
It is also possible to read and modify the console loglevel using the text file /proc/sys/
kernel/printk. The file hosts four integer values: the current loglevel, the default level
for messages that lack an explicit loglevel, the minimum allowed loglevel, and the
boot-time default loglevel. Writing a single value to this file changes the current
loglevel to that value; thus, for example, you can cause all kernel messages to appear
at the console by simply entering:
# echo 8 > /proc/sys/kernel/printk
It should now be apparent why the hello.c sample had the KERN_ALERT; markers; they
are there to make sure that the messages appear on the console.
Redirecting Console Messages
Linux allows for some flexibility in console logging policies by letting you send messages
to a specific virtual console (if your console lives on the text screen). By default,
the "console" is the current virtual terminal. To select a different virtual terminal to
receive messages, you can issue ioctl(TIOCLINUX) on any console device. The following
program, setconsole, can be used to choose which console receives kernel messages;
it must be run by the superuser and is available in the misc-progs directory.
PAGE 78
The following is the program in its entirety. You should invoke it with a single argument
specifying the number of the console that is to receive messages.
int main(int argc, char **argv)
{
char bytes[2] = {11,0}; /* 11 is the TIOCLINUX cmd number */
if (argc==2) bytes[1] = atoi(argv[1]); /* the chosen console */
else {
fprintf(stderr, "%s: need a single arg\n",argv[0]); exit(1);
}
if (ioctl(STDIN_FILENO, TIOCLINUX, bytes)<0) { /* use stdin */
fprintf(stderr,"%s: ioctl(stdin, TIOCLINUX): %s\n",
argv[0], strerror(errno));
exit(1);
}
exit(0);
}
setconsole uses the special ioctl command TIOCLINUX, which implements Linux specific
functions. To use TIOCLINUX, you pass it an argument that is a pointer to a
byte array. The first byte of the array is a number that specifies the requested subcommand,
and the following bytes are subcommand specific. In setconsole, subcommand
11 is used, and the next byte (stored in bytes[1]) identifies the virtual console.
The complete description of TIOCLINUX can be found in drivers/char/tty_io.c, in the
kernel sources.
How Messages Get Logged
The printk function writes messages into a circular buffer that is __LOG_BUF_LEN bytes
long: a value from 4 KB to 1 MB chosen while configuring the kernel. The function
then wakes any process that is waiting for messages, that is, any process that is sleeping
in the syslog system call or that is reading /proc/kmsg. These two interfaces to the
logging engine are almost equivalent, but note that reading from /proc/kmsg consumes
the data from the log buffer, whereas the syslog system call can optionally
return log data while leaving it for other processes as well. In general, reading the
/proc file is easier and is the default behavior for klogd. The dmesg command can be
used to look at the content of the buffer without flushing it; actually, the command
returns to stdout the whole content of the buffer, whether or not it has already been
read.
If you happen to read the kernel messages by hand, after stopping klogd, you'll find
that the /proc file looks like a FIFO, in that the reader blocks, waiting for more data.
Obviously, you can't read messages this way if klogd or another process is already
reading the same data, because you'll contend for it.
If the circular buffer fills up, printk wraps around and starts adding new data to the
beginning of the buffer, overwriting the oldest data. Therefore, the logging process
PAGE 79
loses the oldest data. This problem is negligible compared with the advantages of
using such a circular buffer. For example, a circular buffer allows the system to run
even without a logging process, while minimizing memory waste by overwriting old
data should nobody read it. Another feature of the Linux approach to messaging is
that printk can be invoked from anywhere, even from an interrupt handler, with no
limit on how much data can be printed. The only disadvantage is the possibility of
losing some data.
If the klogd process is running, it retrieves kernel messages and dispatches them to
syslogd, which in turn checks /etc/syslog.conf to find out how to deal with them. syslogd
differentiates between messages according to a facility and a priority; allowable
values for both the facility and the priority are defined in <sys/syslog.h>. Kernel messages
are logged by the LOG_KERN facility at a priority corresponding to the one used in
printk (for example, LOG_ERR is used for KERN_ERR messages). If klogd isn't running,
data remains in the circular buffer until someone reads it or the buffer overflows.
If you want to avoid clobbering your system log with the monitoring messages from
your driver, you can either specify the -f (file) option to klogd to instruct it to save
messages to a specific file, or customize /etc/syslog.conf to suit your needs. Yet
another possibility is to take the brute-force approach: kill klogd and verbosely print
messages on an unused virtual terminal,* or issue the command cat /proc/kmsg from
an unused xterm.
Turning the Messages On and Off
During the early stages of driver development, printk can help considerably in debugging
and testing new code. When you officially release the driver, on the other hand,
you should remove, or at least disable, such print statements. Unfortunately, you're
likely to find that as soon as you think you no longer need the messages and remove
them, you implement a new feature in the driver (or somebody finds a bug), and you
want to turn at least one of the messages back on. There are several ways to solve
both issues, to globally enable or disable your debug messages and to turn individual
messages on or off.
Here we show one way to code printk calls so you can turn them on and off individually
or globally; the technique depends on defining a macro that resolves to a printk
(or printf) call when you want it to:
- Each print statement can be enabled or disabled by removing or adding a single letter to the macro's name.
- All the messages can be disabled at once by changing the value of the CFLAGS
variable before compiling.
* For example, use setlevel 8; setconsole 10 to set up terminal 10 to display messages.
PAGE 80
- The same print statement can be used in kernel code and user-level code, so that
the driver and test programs can be managed in the same way with regard to
extra messages.
The following code fragment implements these features and comes directly from the
header scull.h:
#undef PDEBUG /* undef it, just in case */
#ifdef SCULL_DEBUG
# ifdef __KERNEL__
/* This one if debugging is on, and kernel space */
# define PDEBUG(fmt, args...) printk( KERN_DEBUG "scull: " fmt, ## args)
# else
/* This one for user space */
# define PDEBUG(fmt, args...) fprintf(stderr, fmt, ## args)
# endif
#else
# define PDEBUG(fmt, args...) /* not debugging: nothing */
#endif
#undef PDEBUGG
#define PDEBUGG(fmt, args...) /* nothing: it's a placeholder */
The symbol PDEBUG is defined or undefined, depending on whether SCULL_DEBUG is
defined, and displays information in whatever manner is appropriate to the environment
where the code is running: it uses the kernel call printk when it's in the kernel
and the libc call fprintf to the standard error when run in user space. The PDEBUGG
symbol, on the other hand, does nothing; it can be used to easily "comment" print
statements without removing them entirely.
To simplify the process further, add the following lines to your makefile:
# Comment/uncomment the following line to disable/enable debugging
DEBUG = y
# Add your debugging flag (or not) to CFLAGS
ifeq ($(DEBUG),y)
DEBFLAGS = -O -g -DSCULL_DEBUG # "-O" is needed to expand inlines
else
DEBFLAGS = -O2
endif
CFLAGS += $(DEBFLAGS)
The macros shown in this section depend on a gcc extension to the ANSI C preprocessor
that supports macros with a variable number of arguments. This gcc dependency
shouldn't be a problem, because the kernel proper depends heavily on gcc
features anyway. In addition, the makefile depends on GNU's version of make; once
again, the kernel already depends on GNU make, so this dependency is not a problem.
PAGE 81
If you're familiar with the C preprocessor, you can expand on the given definitions to
implement the concept of a "debug level," defining different levels and assigning an
integer (or bit mask) value to each level to determine how verbose it should be.
But every driver has its own features and monitoring needs. The art of good programming
is in choosing the best trade-off between flexibility and efficiency, and we
can't tell what is the best for you. Remember that preprocessor conditionals (as well
as constant expressions in the code) are executed at compile time, so you must
recompile to turn messages on or off. A possible alternative is to use C conditionals,
which are executed at runtime and, therefore, permit you to turn messaging on and
off during program execution. This is a nice feature, but it requires additional processing
every time the code is executed, which can affect performance even when the
messages are disabled. Sometimes this performance hit is unacceptable.
The macros shown in this section have proven themselves useful in a number of situations,
with the only disadvantage being the requirement to recompile a module after
any changes to its messages.
Rate Limiting
If you are not careful, you can find yourself generating thousands of messages with
printk, overwhelming the console and, possibly, overflowing the system log file.
When using a slow console device (e.g., a serial port), an excessive message rate can
also slow down the system or just make it unresponsive. It can be very hard to get a
handle on what is wrong with a system when the console is spewing out data nonstop.
Therefore, you should be very careful about what you print, especially in production
versions of drivers and especially once initialization is complete. In general,
production code should never print anything during normal operation; printed output
should be an indication of an exceptional situation requiring attention.
On the other hand, you may want to emit a log message if a device you are driving
stops working. But you should be careful not to overdo things. An unintelligent process
that continues forever in the face of failures can generate thousands of retries per
second; if your driver prints a "my device is broken" message every time, it could create
vast amounts of output and possibly hog the CPU if the console device is slow-no
interrupts can be used to driver the console, even if it is a serial port or a line
printer.
In many cases, the best behavior is to set a flag saying, "I have already complained
about this," and not print any further messages once the flag gets set. In others,
though, there are reasons to emit an occasional "the device is still broken" notice.
The kernel has provided a function that can be helpful in such cases:
int printk_ratelimit(void);
PAGE 82
This function should be called before you consider printing a message that could be
repeated often. If the function returns a nonzero value, go ahead and print your message,
otherwise skip it. Thus, typical calls look like this:
if (printk_ratelimit( ))
printk(KERN_NOTICE "The printer is still on fire\n");
printk_ratelimit works by tracking how many messages are sent to the console.
When the level of output exceeds a threshold, printk_ratelimit starts returning 0 and
causing messages to be dropped.
The behavior of printk_ratelimit can be customized by modifying /proc/sys/kernel/
printk_ratelimit (the number of seconds to wait before re-enabling messages) and are
/proc/sys/kernel/printk_ratelimit_burst (the number of messages accepted before rate-limiting).
Printing Device Numbers
Occasionally, when printing a message from a driver, you will want to print the
device number associated with the hardware of interest. It is not particularly hard
to print the major and minor numbers, but, in the interest of consistency, the kernel
provides a couple of utility macros (defined in <linux/kdev_t.h>) for this purpose:
int print_dev_t(char *buffer, dev_t dev);
char *format_dev_t(char *buffer, dev_t dev);
Both macros encode the device number into the given buffer; the only difference is
that print_dev_t returns the number of characters printed, while format_dev_t returns
buffer; therefore, it can be used as a parameter to a printk call directly, although one
must remember that printk doesn't flush until a trailing newline is provided. The
buffer should be large enough to hold a device number; given that 64-bit device
numbers are a distinct possibility in future kernel releases, the buffer should probably
be at least 20 bytes long.
Debugging by Querying
The previous section described how printk works and how it can be used. What it
didn't talk about are its disadvantages.
A massive use of printk can slow down the system noticeably, even if you lower
console_loglevel to avoid loading the console device, because syslogd keeps syncing
its output files; thus, every line that is printed causes a disk operation. This is the
right implementation from syslogd's perspective. It tries to write everything to disk in
case the system crashes right after printing the message; however, you don't want to
slow down your system just for the sake of debugging messages. This problem can be
solved by prefixing the name of your log file as it appears in /etc/syslogd.conf with a
PAGE 83
hyphen.* The problem with changing the configuration file is that the modification
will likely remain there after you are done debugging, even though during normal
system operation you do want messages to be flushed to disk as soon as possible. An
alternative to such a permanent change is running a program other than klogd (such
as cat /proc/kmsg, as suggested earlier), but this may not provide a suitable environment
for normal system operation.
More often than not, the best way to get relevant information is to query the system
when you need the information, instead of continually producing data. In fact, every
Unix system provides many tools for obtaining system information: ps, netstat,
vmstat, and so on.
A few techniques are available to driver developers for querying the system: creating
a file in the /proc filesystem, using the ioctl driver method, and exporting attributes
via sysfs. The use of sysfs requires quite some background on the driver model. It is
discussed in Chapter 14.
Using the /proc Filesystem
The /proc filesystem is a special, software-created filesystem that is used by the kernel
to export information to the world. Each file under /proc is tied to a kernel function
that generates the file's "contents" on the fly when the file is read. We have
already seen some of these files in action; /proc/modules, for example, always returns
a list of the currently loaded modules.
/proc is heavily used in the Linux system. Many utilities on a modern Linux distribution,
such as ps, top, and uptime, get their information from /proc. Some device drivers
also export information via /proc, and yours can do so as well. The /proc
filesystem is dynamic, so your module can add or remove entries at any time.
Fully featured /proc entries can be complicated beasts; among other things, they can
be written to as well as read from. Most of the time, however, /proc entries are readonly
files. This section concerns itself with the simple read-only case. Those who are
interested in implementing something more complicated can look here for the basics;
the kernel source may then be consulted for the full picture.
Before we continue, however, we should mention that adding files under /proc is discouraged.
The /proc filesystem is seen by the kernel developers as a bit of an uncontrolled
mess that has gone far beyond its original purpose (which was to provide
information about the processes running in the system). The recommended way of
making information available in new code is via sysfs. As suggested, working with
sysfs requires an understanding of the Linux device model, however, and we do not
* The hyphen, or minus sign, is a "magic" marker to prevent syslogd from flushing the file to disk at every new
message, documented in syslog.conf(5), a manpage worth reading.
PAGE 84
get to that until Chapter 14. Meanwhile, files under /proc are slightly easier to create,
and they are entirely suitable for debugging purposes, so we cover them here.
Implementing files in /proc
All modules that work with /proc should include <linux/proc_fs.h> to define the
proper functions.
To create a read-only /proc file, your driver must implement a function to produce
the data when the file is read. When some process reads the file (using the read system
call), the request reaches your module by means of this function. We'll look at
this function first and get to the registration interface later in this section.
When a process reads from your /proc file, the kernel allocates a page of memory (i.e.,
PAGE_SIZE bytes) where the driver can write data to be returned to user space. That
buffer is passed to your function, which is a method called read_proc:
int (*read_proc)(char *page, char **start, off_t offset, int count,
int *eof, void *data);
The page pointer is the buffer where you'll write your data; start is used by the function
to say where the interesting data has been written in page (more on this later);
offset and count have the same meaning as for the read method. The eof argument
points to an integer that must be set by the driver to signal that it has no more
data to return, while data is a driver-specific data pointer you can use for internal
bookkeeping.
This function should return the number of bytes of data actually placed in the page
buffer, just like the read method does for other files. Other output values are *eof
and *start. eof is a simple flag, but the use of the start value is somewhat more
complicated; its purpose is to help with the implementation of large (greater than
one page) /proc files.
The start parameter has a somewhat unconventional use. Its purpose is to indicate
where (within page) the data to be returned to the user is found. When your proc_read
method is called, *start will be NULL. If you leave it NULL, the kernel assumes that the
data has been put into page as if offset were 0; in other words, it assumes a simpleminded
version of proc_read, which places the entire contents of the virtual file in page
without paying attention to the offset parameter. If, instead, you set *start to a nonNULL
value, the kernel assumes that the data pointed to by *start takes offset into
account and is ready to be returned directly to the user. In general, simple proc_read
methods that return tiny amounts of data just ignore start. More complex methods set
*start to page and only place data beginning at the requested offset there.
There has long been another major issue with /proc files, which start is meant to
solve as well. Sometimes the ASCII representation of kernel data structures changes
between successive calls to read, so the reader process could find inconsistent data
from one call to the next. If *start is set to a small integer value, the caller uses it to
PAGE 85
increment filp->f_pos independently of the amount of data you return, thus making
f_pos an internal record number of your read_proc procedure. If, for example,
your read_proc function is returning information from a big array of structures, and
five of those structures were returned in the first call, *start could be set to 5. The
next call provides that same value as the offset; the driver then knows to start returning
data from the sixth structure in the array. This is acknowledged as a "hack" by its
authors and can be seen in fs/proc/generic.c.
Note that there is a better way to implement large /proc files; it's called seq_file, and
we'll discuss it shortly. First, though, it is time for an example. Here is a simple (if
somewhat ugly) read_proc implementation for the scull device:
int scull_read_procmem(char *buf, char **start, off_t offset,
int count, int *eof, void *data)
{
int i, j, len = 0;
int limit = count - 80; /* Don't print more than this */
for (i = 0; i < scull_nr_devs && len <= limit; i++) {
struct scull_dev *d = &scull_devices[i];
struct scull_qset *qs = d->data;
if (down_interruptible(&d->sem))
return -ERESTARTSYS;
len += sprintf(buf+len,"\nDevice %i: qset %i, q %i, sz %li\n",
i, d->qset, d->quantum, d->size);
for (; qs && len <= limit; qs = qs->next) { /* scan the list */
len += sprintf(buf + len, " item at %p, qset at %p\n",
qs, qs->data);
if (qs->data && !qs->next) /* dump only the last item */
for (j = 0; j < d->qset; j++) {
if (qs->data[j])
len += sprintf(buf + len,
" % 4i: %8p\n",
j, qs->data[j]);
}
}
up(&scull_devices[i].sem);
}
*eof = 1;
return len;
}
This is a fairly typical read_proc implementation. It assumes that there will never be a
need to generate more than one page of data and so ignores the start and offset values.
It is, however, careful not to overrun its buffer, just in case.
An older interface
If you read through the kernel source, you may encounter code implementing /proc
files with an older interface:
int (*get_info)(char *page, char **start, off_t offset, int count);
PAGE 86
All of the arguments have the same meaning as they do for read_proc, but the eof
and data arguments are missing. This interface is still supported, but it could go
away in the future; new code should use the read_proc interface instead.
Creating your /proc file
Once you have a read_proc function defined, you need to connect it to an entry in
the /proc hierarchy. This is done with a call to create_proc_read_entry:
struct proc_dir_entry *create_proc_read_entry(const char *name,
mode_t mode, struct proc_dir_entry *base,
read_proc_t *read_proc, void *data);
Here, name is the name of the file to create, mode is the protection mask for the file (it
can be passed as 0 for a system-wide default), base indicates the directory in which the
file should be created (if base is NULL, the file is created in the /proc root), read_proc is
the read_proc function that implements the file, and data is ignored by the kernel (but
passed to read_proc). Here is the call used by scull to make its /proc function available
as /proc/scullmem:
create_proc_read_entry("scullmem", 0 /* default mode */,
NULL /* parent dir */, scull_read_procmem,
NULL /* client data */);
Here, we create a file called scullmem directly under /proc, with the default, world-readable
protections.
The directory entry pointer can be used to create entire directory hierarchies under
/proc. Note, however, that an entry may be more easily placed in a subdirectory of
/proc simply by giving the directory name as part of the name of the entry--as long
as the directory itself already exists. For example, an (often ignored) convention says
that /proc entries associated with device drivers should go in the subdirectory driver/;
scull could place its entry there simply by giving its name as driver/scullmem.
Entries in /proc, of course, should be removed when the module is unloaded.
remove_proc_entry is the function that undoes what create_proc_read_entry already
did:
remove_proc_entry("scullmem", NULL /* parent dir */);
Failure to remove entries can result in calls at unwanted times, or, if your module has
been unloaded, kernel crashes.
When using /proc files as shown, you must remember a few nuisances of the implementation--no
surprise its use is discouraged nowadays.
The most important problem is with removal of /proc entries. Such removal may well
happen while the file is in use, as there is no owner associated to /proc entries, so
using them doesn't act on the module's reference count. This problem is simply triggered
by running sleep 100 < /proc/myfile just before removing the module, for example.
PAGE 87
Another issue is about registering two entries with the same name. The kernel trusts
the driver and doesn't check if the name is already registered, so if you are not careful
you might end up with two or more entries with the same name. This is a problem
known to happen in classrooms, and such entries are indistinguishable, both
when you access them and when you call remove_proc_entry.
The seq_file interface
As we noted above, the implementation of large files under /proc is a little awkward.
Over time, /proc methods have become notorious for buggy implementations when
the amount of output grows large. As a way of cleaning up the /proc code and making
life easier for kernel programmers, the seq_file interface was added. This interface
provides a simple set of functions for the implementation of large kernel virtual
files.
The seq_file interface assumes that you are creating a virtual file that steps through
a sequence of items that must be returned to user space. To use seq_file, you must
create a simple "iterator" object that can establish a position within the sequence,
step forward, and output one item in the sequence. It may sound complicated, but,
in fact, the process is quite simple. We'll step through the creation of a /proc file in
the scull driver to show how it is done.
The first step, inevitably, is the inclusion of <linux/seq_file.h>. Then you must create
four iterator methods, called start, next, stop, and show.
The start method is always called first. The prototype for this function is:
void *start(struct seq_file *sfile, loff_t *pos);
The sfile argument can almost always be ignored. pos is an integer position indicating
where the reading should start. The interpretation of the position is entirely up to
the implementation; it need not be a byte position in the resulting file. Since seq_file
implementations typically step through a sequence of interesting items, the position
is often interpreted as a cursor pointing to the next item in the sequence. The scull
driver interprets each device as one item in the sequence, so the incoming pos is simply
an index into the scull_devices array. Thus, the start method used in scull is:
static void *scull_seq_start(struct seq_file *s, loff_t *pos)
{
if (*pos >= scull_nr_devs)
return NULL; /* No more to read */
return scull_devices + *pos;
}
The return value, if non-NULL, is a private value that can be used by the iterator
implementation.
The next function should move the iterator to the next position, returning NULL if
there is nothing left in the sequence. This method's prototype is:
void *next(struct seq_file *sfile, void *v, loff_t *pos);
PAGE 88
Here, v is the iterator as returned from the previous call to start or next, and pos is
the current position in the file. next should increment the value pointed to by pos;
depending on how your iterator works, you might (though probably won't) want to
increment pos by more than one. Here's what scull does:
static void *scull_seq_next(struct seq_file *s, void *v, loff_t *pos)
{
(*pos)++;
if (*pos >= scull_nr_devs)
return NULL;
return scull_devices + *pos;
}
When the kernel is done with the iterator, it calls stop to clean up:
void stop(struct seq_file *sfile, void *v);
The scull implementation has no cleanup work to do, so its stop method is empty.
It is worth noting that the seq_file code, by design, does not sleep or perform other
nonatomic tasks between the calls to start and stop. You are also guaranteed to see
one stop call sometime shortly after a call to start. Therefore, it is safe for your start
method to acquire semaphores or spinlocks. As long as your other seq_file methods
are atomic, the whole sequence of calls is atomic. (If this paragraph does not
make sense to you, come back to it after you've read the next chapter.)
In between these calls, the kernel calls the show method to actually output something
interesting to the user space. This method's prototype is:
int show(struct seq_file *sfile, void *v);
This method should create output for the item in the sequence indicated by the iterator
v. It should not use printk, however; instead, there is a special set of functions for
seq_file output:
int seq_printf(struct seq_file *sfile, const char *fmt, ...);
This is the printf equivalent for seq_file implementations; it takes the usual format
string and additional value arguments. You must also pass it the seq_file
structure given to the show function, however. If seq_printf returns a nonzero
value, it means that the buffer has filled, and output is being discarded. Most
implementations ignore the return value, however.
int seq_putc(struct seq_file *sfile, char c);
int seq_puts(struct seq_file *sfile, const char *s);
These are the equivalents of the user-space putc and puts functions.
int seq_escape(struct seq_file *m, const char *s, const char *esc);
This function is equivalent to seq_puts with the exception that any character in
s that is also found in esc is printed in octal format. A common value for esc is
" \t\n\\", which keeps embedded white space from messing up the output and
possibly confusing shell scripts.
PAGE 89
int seq_path(struct seq_file *sfile, struct vfsmount *m, struct dentry
*dentry, char *esc);
This function can be used for outputting the file name associated with a given
directory entry. It is unlikely to be useful in device drivers; we have included it
here for completeness.
Getting back to our example; the show method used in scull is:
static int scull_seq_show(struct seq_file *s, void *v)
{
struct scull_dev *dev = (struct scull_dev *) v;
struct scull_qset *d;
int i;
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
seq_printf(s, "\nDevice %i: qset %i, q %i, sz %li\n",
(int) (dev - scull_devices), dev->qset,
dev->quantum, dev->size);
for (d = dev->data; d; d = d->next) { /* scan the list */
seq_printf(s, " item at %p, qset at %p\n", d, d->data);
if (d->data && !d->next) /* dump only the last item */
for (i = 0; i < dev->qset; i++) {
if (d->data[i])
seq_printf(s, " % 4i: %8p\n",
i, d->data[i]);
}
}
up(&dev->sem);
return 0;
}
Here, we finally interpret our "iterator" value, which is simply a pointer to a scull_dev
structure.
Now that it has a full set of iterator operations, scull must package them up and
connect them to a file in /proc. The first step is done by filling in a seq_operations
structure:
static struct seq_operations scull_seq_ops = {
.start = scull_seq_start,
.next = scull_seq_next,
.stop = scull_seq_stop,
.show = scull_seq_show
};
With that structure in place, we must create a file implementation that the kernel
understands. We do not use the read_proc method described previously; when using
seq_file, it is best to connect in to /proc at a slightly lower level. That means creating
a file_operations structure (yes, the same structure used for char drivers) implementing
all of the operations needed by the kernel to handle reads and seeks on the
PAGE 90
file. Fortunately, this task is straightforward. The first step is to create an open
method that connects the file to the seq_file operations:
static int scull_proc_open(struct inode *inode, struct file *file)
{
return seq_open(file, &scull_seq_ops);
}
The call to seq_open connects the file structure with our sequence operations
defined above. As it turns out, open is the only file operation we must implement
ourselves, so we can now set up our file_operations structure:
static struct file_operations scull_proc_ops = {
.owner = THIS_MODULE,
.open = scull_proc_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release
};
Here we specify our own open method, but use the canned methods seq_read, seq_
lseek, and seq_release for everything else.
The final step is to create the actual file in /proc:
entry = create_proc_entry("scullseq", 0, NULL);
if (entry)
entry->proc_fops = &scull_proc_ops;
Rather than using create_proc_read_entry, we call the lower-level create_proc_entry,
which has this prototype:
struct proc_dir_entry *create_proc_entry(const char *name,
mode_t mode,
struct proc_dir_entry *parent);
The arguments are the same as their equivalents in create_proc_read_entry: the name
of the file, its protections, and the parent directory.
With the above code, scull has a new /proc entry that looks much like the previous
one. It is superior, however, because it works regardless of how large its output
becomes, it handles seeks properly, and it is generally easier to read and maintain.
We recommend the use of seq_file for the implementation of files that contain more
than a very small number of lines of output.
The ioctl Method
ioctl, which we show you how to use in Chapter 6, is a system call that acts on a file
descriptor; it receives a number that identifies a command to be performed and
(optionally) another argument, usually a pointer. As an alternative to using the /proc
filesystem, you can implement a few ioctl commands tailored for debugging. These
PAGE 91
commands can copy relevant data structures from the driver to user space where you
can examine them.
Using ioctl this way to get information is somewhat more difficult than using /proc,
because you need another program to issue the ioctl and display the results. This program
must be written, compiled, and kept in sync with the module you're testing.
On the other hand, the driver-side code can be easier than what is needed to implement
a /proc file.
There are times when ioctl is the best way to get information, because it runs faster
than reading /proc. If some work must be performed on the data before it's written to
the screen, retrieving the data in binary form is more efficient than reading a text file.
In addition, ioctl doesn't require splitting data into fragments smaller than a page.
Another interesting advantage of the ioctl approach is that information-retrieval commands
can be left in the driver even when debugging would otherwise be disabled.
Unlike a /proc file, which is visible to anyone who looks in the directory (and too
many people are likely to wonder "what that strange file is"), undocumented ioctl
commands are likely to remain unnoticed. In addition, they will still be there should
something weird happen to the driver. The only drawback is that the module will be
slightly bigger.
Debugging by Watching
Sometimes minor problems can be tracked down by watching the behavior of an
application in user space. Watching programs can also help in building confidence
that a driver is working correctly. For example, we were able to feel confident about
scull after looking at how its read implementation reacted to read requests for different
amounts of data.
There are various ways to watch a user-space program working. You can run a
debugger on it to step through its functions, add print statements, or run the program
under strace. Here we'll discuss just the last technique, which is most interesting
when the real goal is examining kernel code.
The strace command is a powerful tool that shows all the system calls issued by a
user-space program. Not only does it show the calls, but it can also show the arguments
to the calls and their return values in symbolic form. When a system call fails,
both the symbolic value of the error (e.g., ENOMEM) and the corresponding string (Out
of memory) are displayed. strace has many command-line options; the most useful of
which are -t to display the time when each call is executed, -T to display the time
spent in the call, -e to limit the types of calls traced, and -o to redirect the output to a
file. By default, strace prints tracing information on stderr.
strace receives information from the kernel itself. This means that a program can be
traced regardless of whether or not it was compiled with debugging support (the -g
PAGE 92
option to gcc) and whether or not it is stripped. You can also attach tracing to a running
process, similar to the way a debugger can connect to a running process and
control it.
The trace information is often used to support bug reports sent to application developers,
but it's also invaluable to kernel programmers. We've seen how driver code
executes by reacting to system calls; strace allows us to check the consistency of
input and output data of each call.
For example, the following screen dump shows (most of) the last lines of running the
command strace ls /dev > /dev/scull0:
open("/dev", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=24576, ...}) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
getdents64(3, /* 141 entries */, 4096) = 4088
[...]
getdents64(3, /* 0 entries */, 4096) = 0
close(3) = 0
[...]
fstat64(1, {st_mode=S_IFCHR|0664, st_rdev=makedev(254, 0), ...}) = 0
write(1, "MAKEDEV\nadmmidi0\nadmmidi1\nadmmid"..., 4096) = 4000
write(1, "b\nptywc\nptywd\nptywe\nptywf\nptyx0\n"..., 96) = 96
write(1, "b\nptyxc\nptyxd\nptyxe\nptyxf\nptyy0\n"..., 4096) = 3904
write(1, "s17\nvcs18\nvcs19\nvcs2\nvcs20\nvcs21"..., 192) = 192
write(1, "\nvcs47\nvcs48\nvcs49\nvcs5\nvcs50\nvc"..., 673) = 673
close(1) = 0
exit_group(0) = ?
It's apparent from the first write call that after ls finished looking in the target directory,
it tried to write 4 KB. Strangely (for ls), only 4000 bytes were written, and the
operation was retried. However, we know that the write implementation in scull
writes a single quantum at a time, so we could have expected the partial write. After
a few steps, everything sweeps through, and the program exits successfully.
As another example, let's read the scull device (using the wc command):
[...]
open("/dev/scull0", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFCHR|0664, st_rdev=makedev(254, 0), ...}) = 0
read(3, "MAKEDEV\nadmmidi0\nadmmidi1\nadmmid"..., 16384) = 4000
read(3, "b\nptywc\nptywd\nptywe\nptywf\nptyx0\n"..., 16384) = 4000
read(3, "s17\nvcs18\nvcs19\nvcs2\nvcs20\nvcs21"..., 16384) = 865
read(3, "", 16384) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
write(1, "8865 /dev/scull0\n", 17) = 17
close(3) = 0
exit_group(0) = ?
As expected, read is able to retrieve only 4000 bytes at a time, but the total amount
of data is the same that was written in the previous example. It's interesting to note
how retries are organized in this example, as opposed to the previous trace. wc is
PAGE 93
optimized for fast reading and, therefore, bypasses the standard library, trying to
read more data with a single system call. You can see from the read lines in the trace
how wc tried to read 16 KB at a time.
Linux experts can find much useful information in the output of strace. If you're put
off by all the symbols, you can limit yourself to watching how the file methods (open,
read, and so on) work with the efile flag.
Personally, we find strace most useful for pinpointing runtime errors from system
calls. Often the perror call in the application or demo program isn't verbose enough
to be useful for debugging, and being able to tell exactly which arguments to which
system call triggered the error can be a great help.
Debugging System Faults
Even if you've used all the monitoring and debugging techniques, sometimes bugs
remain in the driver, and the system faults when the driver is executed. When this
happens, it's important to be able to collect as much information as possible to solve
the problem.
Note that "fault" doesn't mean "panic." The Linux code is robust enough to respond
gracefully to most errors: a fault usually results in the destruction of the current process
while the system goes on working. The system can panic, and it may if a fault
happens outside of a process's context or if some vital part of the system is compromised.
But when the problem is due to a driver error, it usually results only in the
sudden death of the process unlucky enough to be using the driver. The only unrecoverable
damage when a process is destroyed is that some memory allocated to the
process's context is lost; for instance, dynamic lists allocated by the driver through
kmalloc might be lost. However, since the kernel calls the close operation for any
open device when a process dies, your driver can release what was allocated by the
open method.
Even though an oops usually does not bring down the entire system, you may well
find yourself needing to reboot after one happens. A buggy driver can leave hardware
in an unusable state, leave kernel resources in an inconsistent state, or, in the worst
case, corrupt kernel memory in random places. Often you can simply unload your
buggy driver and try again after an oops. If, however, you see anything that suggests
that the system as a whole is not well, your best bet is usually to reboot immediately.
We've already said that when kernel code misbehaves, an informative message is
printed on the console. The next section explains how to decode and use such messages.
Even though they appear rather obscure to the novice, processor dumps are
full of interesting information, often sufficient to pinpoint a program bug without the
need for additional testing.
PAGE 94
Oops Messages
Most bugs show themselves in NULL pointer dereferences or by the use of other incorrect
pointer values. The usual outcome of such bugs is an oops message.
Almost any address used by the processor is a virtual address and is mapped to physical
addresses through a complex structure of page tables (the exceptions are physical
addresses used with the memory management subsystem itself). When an invalid
pointer is dereferenced, the paging mechanism fails to map the pointer to a physical
address, and the processor signals a page fault to the operating system. If the address
is not valid, the kernel is not able to "page in" the missing address; it (usually) generates
an oops if this happens while the processor is in supervisor mode.
An oops displays the processor status at the time of the fault, including the contents
of the CPU registers and other seemingly incomprehensible information. The message
is generated by printk statements in the fault handler (arch/*/kernel/traps.c) and
is dispatched as described earlier in the section "printk."
Let's look at one such message. Here's what results from dereferencing a NULL pointer
on a PC running Version 2.6 of the kernel. The most relevant information here is the
instruction pointer (EIP), the address of the faulty instruction.
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
d083a064
Oops: 0002 [#1]
SMP
CPU: 0
EIP: 0060:[<d083a064>] Not tainted
EFLAGS: 00010246 (2.6.6)
EIP is at faulty_write+0x4/0x10 [faulty]
eax: 00000000 ebx: 00000000 ecx: 00000000 edx: 00000000
esi: cf8b2460 edi: cf8b2480 ebp: 00000005 esp: c31c5f74
ds: 007b es: 007b ss: 0068
Process bash (pid: 2086, threadinfo=c31c4000 task=cfa0a6c0)
Stack: c0150558 cf8b2460 080e9408 00000005 cf8b2480 00000000 cf8b2460 cf8b2460
fffffff7 080e9408 c31c4000 c0150682 cf8b2460 080e9408 00000005 cf8b2480
00000000 00000001 00000005 c0103f8f 00000001 080e9408 00000005 00000005
Call Trace:
[<c0150558>] vfs_write+0xb8/0x130
[<c0150682>] sys_write+0x42/0x70
[<c0103f8f>] syscall_call+0x7/0xb
Code: 89 15 00 00 00 00 c3 90 8d 74 26 00 83 ec 0c b8 00 a6 83 d0
This message was generated by writing to a device owned by the faulty module, a
module built deliberately to demonstrate failures. The implementation of the write
method of faulty.c is trivial:
ssize_t faulty_write (struct file *filp, const char __user *buf, size_t count,
loff_t *pos)
{
PAGE 95
/* make a simple fault by dereferencing a NULL pointer */
*(int *)0 = 0;
return 0;
}
As you can see, what we do here is dereference a NULL pointer. Since 0 is never a valid
pointer value, a fault occurs, which the kernel turns into the oops message shown
earlier. The calling process is then killed.
The faulty module has a different fault condition in its read implementation:
ssize_t faulty_read(struct file *filp, char __user *buf,
size_t count, loff_t *pos)
{
int ret;
char stack_buf[4];
/* Let's try a buffer overflow */
memset(stack_buf, 0xff, 20);
if (count > 4)
count = 4; /* copy 4 bytes to the user */
ret = copy_to_user(buf, stack_buf, count);
if (!ret)
return count;
return ret;
}
This method copies a string into a local variable; unfortunately, the string is longer
than the destination array. The resulting buffer overflow causes an oops when the
function returns. Since the return instruction brings the instruction pointer to
nowhere land, this kind of fault is much harder to trace, and you can get something
such as the following:
EIP: 0010:[<00000000>]
Unable to handle kernel paging request at virtual address ffffffff
printing eip:
ffffffff
Oops: 0000 [#5]
SMP
CPU: 0
EIP: 0060:[<ffffffff>] Not tainted
EFLAGS: 00010296 (2.6.6)
EIP is at 0xffffffff
eax: 0000000c ebx: ffffffff ecx: 00000000 edx: bfffda7c
esi: cf434f00 edi: ffffffff ebp: 00002000 esp: c27fff78
ds: 007b es: 007b ss: 0068
Process head (pid: 2331, threadinfo=c27fe000 task=c3226150)
Stack: ffffffff bfffda70 00002000 cf434f20 00000001 00000286 cf434f00 fffffff7
bfffda70 c27fe000 c0150612 cf434f00 bfffda70 00002000 cf434f20 00000000
00000003 00002000 c0103f8f 00000003 bfffda70 00002000 00002000 bfffda70
Call Trace:
[<c0150612>] sys_read+0x42/0x70
[<c0103f8f>] syscall_call+0x7/0xb
Code: Bad EIP value.
PAGE 96
In this case, we see only part of the call stack (vfs_read and faulty_read are missing),
and the kernel complains about a "bad EIP value." That complaint, and the offending
address (ffffffff) listed at the beginning are both hints that the kernel stack has
been corrupted.
In general, when you are confronted with an oops, the first thing to do is to look at
the location where the problem happened, which is usually listed separately from the
call stack. In the first oops shown above, the relevant line is:
EIP is at faulty_write+0x4/0x10 [faulty]
Here we see that we were in the function faulty_write, which is located in the faulty
module (which is listed in square brackets). The hex numbers indicate that the
instruction pointer was 4 bytes into the function, which appears to be 10 (hex) bytes
long. Often that is enough to figure out what the problem is.
If you need more information, the call stack shows you how you got to where things
fell apart. The stack itself is printed in hex form; with a bit of work, you can often
determine the values of local variables and function parameters from the stack listing.
Experienced kernel developers can benefit from a certain amount of pattern recognition
here; for example, if we look at the stack listing from the faulty_read oops:
Stack: ffffffff bfffda70 00002000 cf434f20 00000001 00000286 cf434f00 fffffff7
bfffda70 c27fe000 c0150612 cf434f00 bfffda70 00002000 cf434f20 00000000
00000003 00002000 c0103f8f 00000003 bfffda70 00002000 00002000 bfffda70
The ffffffff at the top of the stack is part of our string that broke things. On the
x86 architecture, by default, the user-space stack starts just below 0xc0000000; thus,
the recurring value 0xbfffda70 is probably a user-space stack address; it is, in fact,
the address of the buffer passed to the read system call, replicated each time it is
passed down the kernel call chain. On the x86 (again, by default), kernel space starts
at 0xc0000000, so values above that are almost certainly kernel-space addresses, and
so on.
Finally, when looking at oops listings, always be on the lookout for the "slab poisoning"
values discussed at the beginning of this chapter. Thus, for example, if you get a
kernel oops where the offending address is 0xa5a5a5a5, you are almost certainly forgetting
to initialize dynamic memory somewhere.
Please note that you see a symbolic call stack (as shown above) only if your kernel is
built with the CONFIG_KALLSYMS option turned on. Otherwise, you see a bare, hexadecimal
listing, which is far less useful until you have decoded it in other ways.
System Hangs
Although most bugs in kernel code end up as oops messages, sometimes they can
completely hang the system. If the system hangs, no message is printed. For example,
PAGE 97
if the code enters an endless loop, the kernel stops scheduling,* and the system doesn't
respond to any action, including the magic Ctrl-Alt-Del combination. You have two
choices for dealing with system hangs--either prevent them beforehand or be able to
debug them after the fact.
You can prevent an endless loop by inserting schedule invocations at strategic points.
The schedule call (as you might guess) invokes the scheduler and, therefore, allows
other processes to steal CPU time from the current process. If a process is looping in
kernel space due to a bug in your driver, the schedule calls enable you to kill the process
after tracing what is happening.
You should be aware, of course, that any call to schedule may create an additional
source of reentrant calls to your driver, since it allows other processes to run. This
reentrancy should not normally be a problem, assuming that you have used suitable
locking in your driver. Be sure, however, not to call schedule any time that your
driver is holding a spinlock.
If your driver really hangs the system, and you don't know where to insert schedule
calls, the best way to go may be to add some print messages and write them to the
console (by changing the console_loglevel value if need be).
Sometimes the system may appear to be hung, but it isn't. This can happen, for
example, if the keyboard remains locked in some strange way. These false hangs can
be detected by looking at the output of a program you keep running for just this purpose.
A clock or system load meter on your display is a good status monitor; as long
as it continues to update, the scheduler is working.
An indispensable tool for many lockups is the "magic SysRq key," which is available
on most architectures. Magic SysRq is invoked with the combination of the Alt and
SysRq keys on the PC keyboard, or with other special keys on other platforms (see
Documentation/sysrq.txt for details), and is available on the serial console as well. A
third key, pressed along with these two, performs one of a number of useful actions:
- r
- Turns off keyboard raw mode; useful in situations where a crashed application
(such as the X server) may have left your keyboard in a strange state.
- k
- Invokes the "secure attention key" (SAK) function. SAK kills all processes running
on the current console, leaving you with a clean terminal.
- s
- Performs an emergency synchronization of all disks.
- u
- Umount. Attempts to remount all disks in a read-only mode. This operation,
usually invoked immediately after s, can save a lot of filesystem checking time in
cases where the system is in serious trouble.
* Actually, multiprocessor systems still schedule on the other processors, and even a uniprocessor machine
might reschedule if kernel preemption is enabled. For the most common case (uniprocessor with preemption
disabled), however, the system stops scheduling altogether.
PAGE 98
- b
- Boot. Immediately reboots the system. Be sure to synchronize and remount the
disks first.
- p
- Prints processor registers information.
- t
- Prints the current task list.
- m
- Prints memory information.
Other magic SysRq functions exist; see sysrq.txt in the Documentation directory of
the kernel source for the full list. Note that magic SysRq must be explicitly enabled in
the kernel configuration and that most distributions do not enable it, for obvious
security reasons. For a system used to develop drivers, however, enabling magic
SysRq is worth the trouble of building a new kernel in itself. Magic SysRq may be
disabled at runtime with a command such as the following:
echo 0 > /proc/sys/kernel/sysrq
You should consider disabling it if unprivileged users can reach your system keyboard,
to prevent accidental or willing damages. Some previous kernel versions had
sysrq disabled by default, so you needed to enable it at runtime by writing 1 to that
same /proc/sys file.
The sysrq operations are exceedingly useful, so they have been made available to system
administrators who can't reach the console. The file /proc/sysrq-trigger is a write only
entry point, where you can trigger a specific sysrq action by writing the associated
command character; you can then collect any output data from the kernel logs.
This entry point to sysrq is always working, even if sysrq is disabled on the console.
If you are experiencing a "live hang," in which your driver is stuck in a loop but the
system as a whole is still functioning, there are a couple of techniques worth knowing.
Often, the SysRq p function points the finger directly at the guilty routine. Failing
that, you can also use the kernel profiling function. Build a kernel with profiling
enabled, and boot it with profile=2 on the command line. Reset the profile counters
with the readprofile utility, then send your driver into its loop. After a little while, use
readprofile again to see where the kernel is spending its time. Another more
advanced alternative is oprofile, that you may consider as well. The file
Documentation/basic_profiling.txt tells you everything you need to know to get
started with the profilers.
One precaution worth using when chasing system hangs is to mount all your disks as
read-only (or unmount them). If the disks are read-only or unmounted, there's no
risk of damaging the filesystem or leaving it in an inconsistent state. Another possibility
is using a computer that mounts all of its filesystems via NFS, the network file
system. The "NFS-Root" capability must be enabled in the kernel, and special
parameters must be passed at boot time. In this case, you'll avoid filesystem corruption
without even resorting to SysRq, because filesystem coherence is managed by
the NFS server, which is not brought down by your device driver.
PAGE 99
Debuggers and Related Tools
The last resort in debugging modules is using a debugger to step through the code,
watching the value of variables and machine registers. This approach is time-consuming
and should be avoided whenever possible. Nonetheless, the fine-grained perspective
on the code that is achieved through a debugger is sometimes invaluable.
Using an interactive debugger on the kernel is a challenge. The kernel runs in its own
address space on behalf of all the processes on the system. As a result, a number of
common capabilities provided by user-space debuggers, such as breakpoints and single-stepping,
are harder to come by in the kernel. In this section we look at several
ways of debugging the kernel; each of them has advantages and disadvantages.
Using gdb
gdb can be quite useful for looking at the system internals. Proficient use of the
debugger at this level requires some confidence with gdb commands, some understanding
of assembly code for the target platform, and the ability to match source
code and optimized assembly.
The debugger must be invoked as though the kernel were an application. In addition
to specifying the filename for the ELF kernel image, you need to provide the name of
a core file on the command line. For a running kernel, that core file is the kernel core
image, /proc/kcore. A typical invocation of gdb looks like the following:
gdb /usr/src/linux/vmlinux /proc/kcore
The first argument is the name of the uncompressed ELF kernel executable, not the
zImage or bzImage or anything built specifically for the boot environment.
The second argument on the gdb command line is the name of the core file. Like any
file in /proc, /proc/kcore is generated when it is read. When the read system call executes
in the /proc filesystem, it maps to a data-generation function rather than a data retrieval
one; we've already exploited this feature in the section "Using the /proc
Filesystem" earlier in this chapter. kcore is used to represent the kernel "executable"
in the format of a core file; it is a huge file, because it represents the whole kernel
address space, which corresponds to all physical memory. From within gdb, you can
look at kernel variables by issuing the standard gdb commands. For example, p jiffies
prints the number of clock ticks from system boot to the current time.
When you print data from gdb, the kernel is still running, and the various data items
have different values at different times; gdb, however, optimizes access to the core
file by caching data that has already been read. If you try to look at the jiffies variable
once again, you'll get the same answer as before. Caching values to avoid extra
disk access is a correct behavior for conventional core files but is inconvenient when
a "dynamic" core image is used. The solution is to issue the command core-file /proc/
kcore whenever you want to flush the gdb cache; the debugger gets ready to use a
PAGE 100
new core file and discards any old information. You won't, however, always need to
issue core-file when reading a new datum; gdb reads the core in chunks of a few kilobytes
and caches only chunks it has already referenced.
Numerous capabilities normally provided by gdb are not available when you are
working with the kernel. For example, gdb is not able to modify kernel data; it
expects to be running a program to be debugged under its own control before playing
with its memory image. It is also not possible to set breakpoints or watchpoints,
or to single-step through kernel functions.
Note that, in order to have symbol information available for gdb, you must compile
your kernel with the CONFIG_DEBUG_INFO option set. The result is a far larger kernel
image on disk, but, without that information, digging through kernel variables is
almost impossible.
With the debugging information available, you can learn a lot about what is going on
inside the kernel. gdb happily prints out structures, follows pointers, etc. One thing
that is harder, however, is examining modules. Since modules are not part of the
vmlinux image passed to gdb, the debugger knows nothing about them. Fortunately,
as of kernel 2.6.7, it is possible to teach gdb what it needs to know to examine loadable
modules.
Linux loadable modules are ELF-format executable images; as such, they have been
divided up into numerous sections. A typical module can contain a dozen or more
sections, but there are typically three that are relevant in a debugging session:
.text
This section contains the executable code for the module. The debugger must
know where this section is to be able to give trace-backs or set breakpoints. (Neither
of these operations is relevant when running the debugger on /proc/kcore,
but they can useful when working with kgdb, described below).
.bss
.data
These two sections hold the module's variables. Any variable that is not initialized
at compile time ends up in .bss, while those that are initialized go into .data.
Making gdb work with loadable modules requires informing the debugger about
where a given module's sections have been loaded. That information is available in
sysfs, under /sys/module. For example, after loading the scull module, the directory
/sys/module/scull/sections contains files with names such as .text; the content of each
file is the base address for that section.
We are now in a position to issue a gdb command telling it about our module. The
command we need is add-symbol-file; this command takes as parameters the name
of the module object file, the .text base address, and a series of optional parameters
PAGE 101
describing where any other sections of interest have been put. After digging through
the module section data in sysfs, we can construct a command such as:
(gdb) add-symbol-file .../scull.ko 0xd0832000 \
-s .bss 0xd0837100 \
-s .data 0xd0836be0
We have included a small script in the sample source (gdbline) that can create this
command for a given module.
We can now use gdb to examine variables in our loadable module. Here is a quick
example taken from a scull debugging session:
(gdb) add-symbol-file scull.ko 0xd0832000 \
-s .bss 0xd0837100 \
-s .data 0xd0836be0
add symbol table from file "scull.ko" at
.text_addr = 0xd0832000
.bss_addr = 0xd0837100
.data_addr = 0xd0836be0
(y or n) y
Reading symbols from scull.ko...done.(gdb)
p scull_devices[0]
$1 = {data = 0xcfd66c50,
quantum = 4000,
qset = 1000,
size = 20881,
access_key = 0,
...}
Here we see that the first scull device currently holds 20,881 bytes. If we wanted, we
could follow the data chain, or look at anything else of interest in the module.
One other useful trick worth knowing about is this:
(gdb) print *(address)
Here, fill in a hex address for address; the output is a file and line number for the
code corresponding to that address. This technique may be useful, for example, to
find out where a function pointer really points.
We still cannot perform typical debugging tasks like setting breakpoints or modifying
data; to perform those operations, we need to use a tool like kdb (described next)
or kgdb (which we get to shortly).
The kdb Kernel Debugger
Many readers may be wondering why the kernel does not have any more advanced
debugging features built into it. The answer, quite simply, is that Linus does not
believe in interactive debuggers. He fears that they lead to poor fixes, those which
patch up symptoms rather than addressing the real cause of problems. Thus, no
built-in debuggers.
PAGE 102
Other kernel developers, however, see an occasional use for interactive debugging
tools. One such tool is the kdb built-in kernel debugger, available as a nonofficial
patch from oss.sgi.com. To use kdb, you must obtain the patch (be sure to get a version
that matches your kernel version), apply it, and rebuild and reinstall the kernel.
Note that, as of this writing, kdb works only on IA-32 (x86) systems (though a version
for the IA-64 existed for a while in the mainline kernel source before being
removed).
Once you are running a kdb-enabled kernel, there are a couple of ways to enter the
debugger. Pressing the Pause (or Break) key on the console starts up the debugger.
kdb also starts up when a kernel oops happens or when a breakpoint is hit. In any
case, you see a message that looks something like this:
Entering kdb (0xc0347b80) on processor 0 due to Keyboard Entry
[0]kdb>
Note that just about everything the kernel does stops when kdb is running. Nothing
else should be running on a system where you invoke kdb; in particular, you should
not have networking turned on--unless, of course, you are debugging a network
driver. It is generally a good idea to boot the system in single-user mode if you will be
using kdb.
As an example, consider a quick scull debugging session. Assuming that the driver is
already loaded, we can tell kdb to set a breakpoint in scull_read as follows:
[0]kdb> bp scull_read
Instruction(i) BP #0 at 0xcd087c5dc (scull_read)
is enabled globally adjust 1
[0]kdb> go
The bp command tells kdb to stop the next time the kernel enters scull_read. You
then type go to continue execution. After putting something into one of the scull
devices, we can attempt to read it by running cat under a shell on another terminal,
yielding the following:
Instruction(i) breakpoint #0 at 0xd087c5dc (adjusted)
0xd087c5dc scull_read: int3
Entering kdb (current=0xcf09f890, pid 1575) on processor 0 due to
Breakpoint @ 0xd087c5dc
[0]kdb>
We are now positioned at the beginning of scull_read. To see how we got there, we
can get a stack trace:
[0]kdb> bt
ESP EIP Function (args)
0xcdbddf74 0xd087c5dc [scull]scull_read
0xcdbddf78 0xc0150718 vfs_read+0xb8
0xcdbddfa4 0xc01509c2 sys_read+0x42
0xcdbddfc4 0xc0103fcf syscall_call+0x7
[0]kdb>
PAGE 103
kdb attempts to print out the arguments to every function in the call trace. It gets
confused, however, by optimization tricks used by the compiler. Therefore, it fails to
print the arguments to scull_read.
Time to look at some data. The mds command manipulates data; we can query the
value of the scull_devices pointer with a command such as:
[0]kdb> mds scull_devices 1
0xd0880de8 cf36ac00 ....
Here we asked for one (4-byte) word of data starting at the location of scull_devices;
the answer tells us that our device array is at the address 0xd0880de8; the first device
structure itself is at 0xcf36ac00. To look at that device structure, we need to use that
address:
[0]kdb> mds cf36ac00
0xcf36ac00 ce137dbc ....
0xcf36ac04 00000fa0 ....
0xcf36ac08 000003e8 ....
0xcf36ac0c 0000009b ....
0xcf36ac10 00000000 ....
0xcf36ac14 00000001 ....
0xcf36ac18 00000000 ....
0xcf36ac1c 00000001 ....
The eight lines here correspond to the beginning part of the scull_dev structure.
Therefore, we see that the memory for the first device is allocated at 0xce137dbc, the
quantum is 4000 (hex fa0), the quantum set size is 1000 (hex 3e8), and there are currently
155 (hex 9b) bytes stored in the device.
kdb can change data as well. Suppose we wanted to trim some of the data from the
device:
[0]kdb> mm cf36ac0c 0x50
0xcf26ac0c = 0x50
A subsequent cat on the device will now return less data than before.
kdb has a number of other capabilities, including single-stepping (by instructions, not
lines of C source code), setting breakpoints on data access, disassembling code, stepping
through linked lists, accessing register data, and more. After you have applied the
kdb patch, a full set of manual pages can be found in the Documentation/kdb directory
in your kernel source tree.
The kgdb Patches
The two interactive debugging approaches we have looked at so far (using gdb on
/proc/kcore and kdb) both fall short of the sort of environment that user-space application
developers have become used to. Wouldn't it be nice if there were a true
debugger for the kernel that supported features like changing variables, breakpoints,
etc.?
PAGE 104
As it turns out, such a solution does exist. There are, as of this writing, two separate
patches in circulation that allow gdb, with full capabilities, to be run against the kernel.
Confusingly, both of these patches are called kgdb. They work by separating the
system running the test kernel from the system running the debugger; the two are
typically connected via a serial cable. Therefore, the developer can run gdb on his or
her stable desktop system, while operating on a kernel running on a sacrificial test
box. Setting up gdb in this mode takes a little time at the outset, but that investment
can pay off quickly when a difficult bug shows up.
These patches are in a strong state of flux, and may even be merged at some point, so
we avoid saying much about them beyond where they are and their basic features.
Interested readers are encouraged to look and see the current state of affairs.
The first kgdb patch is currently found in the -mm kernel tree--the staging area for
patches on their way into the 2.6 mainline. This version of the patch supports the
x86, SuperH, ia64, x86_64, SPARC, and 32-bit PPC architectures. In addition to the
usual mode of operation over a serial port, this version of kgdb can also communicate
over a local-area network. It is simply a matter of enabling the Ethernet mode
and booting with the kgdboe parameter set to indicate the IP address from which
debugging commands can originate. The documentation under Documentation/i386/
kgdb describes how to set things up.*
As an alternative, you can use the kgdb patch found on http://kgdb.sf.net/. This version
of the debugger does not support the network communication mode (though
that is said to be under development), but it does have some built-in support for
working with loadable modules. It supports the x86, x86_64, PowerPC, and S/390
architectures.
The User-Mode Linux Port
User-Mode Linux (UML) is an interesting concept. It is structured as a separate port
of the Linux kernel with its own arch/um subdirectory. It does not run on a new type
of hardware, however; instead, it runs on a virtual machine implemented on the
Linux system call interface. Thus, UML allows the Linux kernel to run as a separate,
user-mode process on a Linux system.
Having a copy of the kernel running as a user-mode process brings a number of
advantages. Because it is running on a constrained, virtual processor, a buggy kernel
cannot damage the "real" system. Different hardware and software configurations can
be tried easily on the same box. And, perhaps most significantly for kernel developers,
the user-mode kernel can be easily manipulated with gdb or another debugger.
* It does neglect to point out that you should have your network adapter driver built into the kernel, however,
or the debugger fails to find it at boot time and will shut itself down.
PAGE 105
After all, it is just another process. UML clearly has the potential to accelerate kernel
development.
However, UML has a big shortcoming from the point of view of driver writers: the
user-mode kernel has no access to the host system's hardware. Thus, while it can be
useful for debugging most of the sample drivers in this book, UML is not yet useful
for debugging drivers that have to deal with real hardware.
See http://user-mode-linux.sf.net/ for more information on UML.
The Linux Trace Toolkit
The Linux Trace Toolkit (LTT) is a kernel patch and a set of related utilities that
allow the tracing of events in the kernel. The trace includes timing information and
can create a reasonably complete picture of what happened over a given period of
time. Thus, it can be used not only for debugging but also for tracking down performance
problems.
LTT, along with extensive documentation, can be found at http://www.opersys.com/LTT.
Dynamic Probes
Dynamic Probes (or DProbes) is a debugging tool released (under the GPL) by IBM
for Linux on the IA-32 architecture. It allows the placement of a "probe" at almost
any place in the system, in both user and kernel space. The probe consists of some
code (written in a specialized, stack-oriented language) that is executed when control
hits the given point. This code can report information back to user space, change
registers, or do a number of other things. The useful feature of DProbes is that once
the capability has been built into the kernel, probes can be inserted anywhere within
a running system without kernel builds or reboots. DProbes can also work with the
LTT to insert new tracing events at arbitrary locations.
The DProbes tool can be downloaded from IBM's open source site: http://oss.software.ibm.com.
PAGE 106
CHAPTER 5
Concurrency and Race Conditions
Thus far, we have paid little attention to the problem of concurrency--i.e., what
happens when the system tries to do more than one thing at once. The management
of concurrency is, however, one of the core problems in operating systems programming.
Concurrency-related bugs are some of the easiest to create and some of the
hardest to find. Even expert Linux kernel programmers end up creating concurrency-related
bugs on occasion.
In early Linux kernels, there were relatively few sources of concurrency. Symmetric
multiprocessing (SMP) systems were not supported by the kernel, and the only cause
of concurrent execution was the servicing of hardware interrupts. That approach
offers simplicity, but it no longer works in a world that prizes performance on systems
with more and more processors, and that insists that the system respond to
events quickly. In response to the demands of modern hardware and applications,
the Linux kernel has evolved to a point where many more things are going on simultaneously.
This evolution has resulted in far greater performance and scalability. It
has also, however, significantly complicated the task of kernel programming. Device
driver programmers must now factor concurrency into their designs from the beginning,
and they must have a strong understanding of the facilities provided by the kernel
for concurrency management.
The purpose of this chapter is to begin the process of creating that understanding.
To that end, we introduce facilities that are immediately applied to the scull driver
from Chapter 3. Other facilities presented here are not put to use for some time yet.
But first, we take a look at what could go wrong with our simple scull driver and how
to avoid these potential problems.
PAGE 107
Pitfalls in scull
Let us take a quick look at a fragment of the scull memory management code. Deep
down inside the write logic, scull must decide whether the memory it requires has
been allocated yet or not. One piece of the code that handles this task is:
if (!dptr->data[s_pos]) {
dptr->data[s_pos] = kmalloc(quantum, GFP_KERNEL);
if (!dptr->data[s_pos])
goto out;
}
Suppose for a moment that two processes (we'll call them "A" and "B") are independently
attempting to write to the same offset within the same scull device. Each process
reaches the if test in the first line of the fragment above at the same time. If the
pointer in question is NULL, each process will decide to allocate memory, and each
will assign the resulting pointer to dptr->data[s_pos]. Since both processes are
assigning to the same location, clearly only one of the assignments will prevail.
What will happen, of course, is that the process that completes the assignment second
will "win." If process A assigns first, its assignment will be overwritten by process
B. At that point, scull will forget entirely about the memory that A allocated; it
only has a pointer to B's memory. The memory allocated by A, thus, will be dropped
and never returned to the system.
This sequence of events is a demonstration of a race condition. Race conditions are a
result of uncontrolled access to shared data. When the wrong access pattern happens,
something unexpected results. For the race condition discussed here, the result
is a memory leak. That is bad enough, but race conditions can often lead to system
crashes, corrupted data, or security problems as well. Programmers can be tempted
to disregard race conditions as extremely low probability events. But, in the computing
world, one-in-a-million events can happen every few seconds, and the consequences
can be grave.
We will eliminate race conditions from scull shortly, but first we need to take a more
general view of concurrency.
Concurrency and Its Management
In a modern Linux system, there are numerous sources of concurrency and, therefore,
possible race conditions. Multiple user-space processes are running, and they
can access your code in surprising combinations of ways. SMP systems can be executing
your code simultaneously on different processors. Kernel code is preemptible;
your driver's code can lose the processor at any time, and the process that replaces it
could also be running in your driver. Device interrupts are asynchronous events that
can cause concurrent execution of your code. The kernel also provides various mechanisms
for delayed code execution, such as workqueues, tasklets, and timers, which
PAGE 108
can cause your code to run at any time in ways unrelated to what the current process
is doing. In the modern, hot-pluggable world, your device could simply disappear
while you are in the middle of working with it.
Avoidance of race conditions can be an intimidating task. In a world where anything
can happen at any time, how does a driver programmer avoid the creation of absolute
chaos? As it turns out, most race conditions can be avoided through some
thought, the kernel's concurrency control primitives, and the application of a few
basic principles. We'll start with the principles first, then get into the specifics of
how to apply them.
Race conditions come about as a result of shared access to resources. When two
threads of execution* have a reason to work with the same data structures (or hardware
resources), the potential for mixups always exists. So the first rule of thumb to
keep in mind as you design your driver is to avoid shared resources whenever possible.
If there is no concurrent access, there can be no race conditions. So carefully written
kernel code should have a minimum of sharing. The most obvious application
of this idea is to avoid the use of global variables. If you put a resource in a place
where more than one thread of execution can find it, there should be a strong reason
for doing so.
The fact of the matter is, however, that such sharing is often required. Hardware
resources are, by their nature, shared, and software resources also must often be
available to more than one thread. Bear in mind as well that global variables are far
from the only way to share data; any time your code passes a pointer to some other
part of the kernel, it is potentially creating a new sharing situation. Sharing is a fact
of life.
Here is the hard rule of resource sharing: any time that a hardware or software
resource is shared beyond a single thread of execution, and the possibility exists that
one thread could encounter an inconsistent view of that resource, you must explicitly
manage access to that resource. In the scull example above, process B's view of
the situation is inconsistent; unaware that process A has already allocated memory
for the (shared) device, it performs its own allocation and overwrites A's work. In
this case, we must control access to the scull data structure. We need to arrange
things so that the code either sees memory that has been allocated or knows that no
memory has been or will be allocated by anybody else. The usual technique for
access management is called locking or mutual exclusion--making sure that only one
thread of execution can manipulate a shared resource at any time. Much of the rest
of this chapter will be devoted to locking.
* For the purposes of this chapter, a "thread" of execution is any context that is running code. Each process is
clearly a thread of execution, but so is an interrupt handler or other code running in response to an asynchronous
kernel event.
PAGE 109
First, however, we must briefly consider one other important rule. When kernel code
creates an object that will be shared with any other part of the kernel, that object
must continue to exist (and function properly) until it is known that no outside references
to it exist. The instant that scull makes its devices available, it must be prepared
to handle requests on those devices. And scull must continue to be able to
handle requests on its devices until it knows that no reference (such as open userspace
files) to those devices exists. Two requirements come out of this rule: no object
can be made available to the kernel until it is in a state where it can function properly,
and references to such objects must be tracked. In most cases, you'll find that
the kernel handles reference counting for you, but there are always exceptions.
Following the above rules requires planning and careful attention to detail. It is easy
to be surprised by concurrent access to resources you hadn't realized were shared.
With some effort, however, most race conditions can be headed off before they bite
you--or your users.
Semaphores and Mutexes
So let us look at how we can add locking to scull. Our goal is to make our operations
on the scull data structure atomic, meaning that the entire operation happens at once
as far as other threads of execution are concerned. For our memory leak example, we
need to ensure that if one thread finds that a particular chunk of memory must be
allocated, it has the opportunity to perform that allocation before any other thread
can make that test. To this end, we must set up critical sections: code that can be executed
by only one thread at any given time.
Not all critical sections are the same, so the kernel provides different primitives for
different needs. In this case, every access to the scull data structure happens in process
context as a result of a direct user request; no accesses will be made from interrupt
handlers or other asynchronous contexts. There are no particular latency
(response time) requirements; application programmers understand that I/O
requests are not usually satisfied immediately. Furthermore, the scull is not holding
any other critical system resource while it is accessing its own data structures. What
all this means is that if the scull driver goes to sleep while waiting for its turn to
access the data structure, nobody is going to mind.
"Go to sleep" is a well-defined term in this context. When a Linux process reaches a
point where it cannot make any further processes, it goes to sleep (or "blocks"),
yielding the processor to somebody else until some future time when it can get work
done again. Processes often sleep when waiting for I/O to complete. As we get
deeper into the kernel, we will encounter a number of situations where we cannot
sleep. The write method in scull is not one of those situations, however. So we can
use a locking mechanism that might cause the process to sleep while waiting for
access to the critical section.
PAGE 110
Just as importantly, we will be performing an operation (memory allocation with
kmalloc) that could sleep--so sleeps are a possibility in any case. If our critical sections
are to work properly, we must use a locking primitive that works when a thread
that owns the locksleeps. Not all locking mechanisms can be used where sleeping is
a possibility (we'll see some that don't later in this chapter). For our present needs,
however, the mechanism that fits best is a semaphore.
Semaphores are a well-understood concept in computer science. At its core, a semaphore
is a single integer value combined with a pair of functions that are typically
called P and V. A process wishing to enter a critical section will call P on the relevant
semaphore; if the semaphore's value is greater than zero, that value is decremented
by one and the process continues. If, instead, the semaphore's value is 0 (or less), the
process must wait until somebody else releases the semaphore. Unlocking a semaphore
is accomplished by calling V; this function increments the value of the semaphore
and, if necessary, wakes up processes that are waiting.
When semaphores are used for mutual exclusion--keeping multiple processes from
running within a critical section simultaneously--their value will be initially set to 1.
Such a semaphore can be held only by a single process or thread at any given time. A
semaphore used in this mode is sometimes called a mutex, which is, of course, an
abbreviation for "mutual exclusion." Almost all semaphores found in the Linux kernel
are used for mutual exclusion.
The Linux Semaphore Implementation
The Linux kernel provides an implementation of semaphores that conforms to the
above semantics, although the terminology is a little different. To use semaphores,
kernel code must include <asm/semaphore.h>. The relevant type is struct semaphore;
actual semaphores can be declared and initialized in a few ways. One is to create a
semaphore directly, then set it up with sema_init:
void sema_init(struct semaphore *sem, int val);
where val is the initial value to assign to a semaphore.
Usually, however, semaphores are used in a mutex mode. To make this common
case a little easier, the kernel has provided a set of helper functions and macros.
Thus, a mutex can be declared and initialized with one of the following:
DECLARE_MUTEX(name);
DECLARE_MUTEX_LOCKED(name);
Here, the result is a semaphore variable (called name) that is initialized to 1 (with
DECLARE_MUTEX) or 0 (with DECLARE_MUTEX_LOCKED). In the latter case, the mutex starts
out in a locked state; it will have to be explicitly unlocked before any thread will be
allowed access.
PAGE 111
If the mutex must be initialized at runtime (which is the case if it is allocated dynamically,
for example), use one of the following:
void init_MUTEX(struct semaphore *sem);
void init_MUTEX_LOCKED(struct semaphore *sem);
In the Linux world, the P function is called down--or some variation of that name.
Here, "down" refers to the fact that the function decrements the value of the semaphore
and, perhaps after putting the caller to sleep for a while to wait for the semaphore
to become available, grants access to the protected resources. There are three
versions of down:
void down(struct semaphore *sem);
int down_interruptible(struct semaphore *sem);
int down_trylock(struct semaphore *sem);
down decrements the value of the semaphore and waits as long as need
be. down_interruptible does the same, but the operation is interruptible. The interruptible version
is almost always the one you will want; it allows a user-space process that is
waiting on a semaphore to be interrupted by the user. You do not, as a general rule,
want to use noninterruptible operations unless there truly is no alternative. Noninterruptible
operations are a good way to create unkillable processes (the dreaded
"D state" seen in ps), and annoy your users. Using down_interruptible requires some
extra care, however, if the operation is interrupted, the function returns a nonzero
value, and the caller does not hold the semaphore. Proper use of down_interruptible
requires always checking the return value and responding accordingly.
The final version (down_trylock) never sleeps; if the semaphore is not available at the
time of the call, down_trylock returns immediately with a nonzero return value.
Once a thread has successfully called one of the versions of down, it is said to be
"holding" the semaphore (or to have "taken out" or "acquired" the semaphore).
That thread is now entitled to access the critical section protected by the semaphore.
When the operations requiring mutual exclusion are complete, the semaphore must
be returned. The Linux equivalent to V is up:
void up(struct semaphore *sem);
Once up has been called, the caller no longer holds the semaphore.
As you would expect, any thread that takes out a semaphore is required to release it
with one (and only one) call to up. Special care is often required in error paths; if an
error is encountered while a semaphore is held, that semaphore must be released
before returning the error status to the caller. Failure to free a semaphore is an easy
error to make; the result (processes hanging in seemingly unrelated places) can be
hard to reproduce and track down.
PAGE 112
Using Semaphores in scull
The semaphore mechanism gives scull a tool that can be used to avoid race conditions
while accessing the scull_dev data structure. But it is up to us to use that tool
correctly. The keys to proper use of locking primitives are to specify exactly which
resources are to be protected and to make sure that every access to those resources
uses the proper locking. In our example driver, everything of interest is contained
within the scull_dev structure, so that is the logical scope for our locking regime.
Let's look again at that structure:
struct scull_dev {
struct scull_qset *data; /* Pointer to first quantum set */
int quantum; /* the current quantum size */
int qset; /* the current array size */
unsigned long size; /* amount of data stored here */
unsigned int access_key; /* used by sculluid and scullpriv */
struct semaphore sem; /* mutual exclusion semaphore */
struct cdev cdev; /* Char device structure */
};
Toward the bottom of the structure is a member called sem which is, of course, our
semaphore. We have chosen to use a separate semaphore for each virtual scull
device. It would have been equally correct to use a single, global semaphore. The various
scull devices share no resources in common, however, and there is no reason to
make one process wait while another process is working with a different scull device.
Using a separate semaphore for each device allows operations on different devices to
proceed in parallel and, therefore, improves performance.
Semaphores must be initialized before use. scull performs this initialization at load
time in this loop:
for (i = 0; i < scull_nr_devs; i++) {
scull_devices[i].quantum = scull_quantum;
scull_devices[i].qset = scull_qset;
init_MUTEX(&scull_devices[i].sem);
scull_setup_cdev(&scull_devices[i], i);
}
Note that the semaphore must be initialized before the scull device is made available
to the rest of the system. Therefore, init_MUTEX is called before scull_setup_cdev.
Performing these operations in the opposite order would create a race condition
where the semaphore could be accessed before it is ready.
Next, we must go through the code and make sure that no accesses to the scull_dev
data structure are made without holding the semaphore. Thus, for example, scull_write
begins with this code:
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
PAGE 113
Note the check on the return value of down_interruptible; if it returns nonzero, the operation
was interrupted. The usual thing to do in this situation is to return -ERESTARTSYS.
Upon seeing this return code, the higher layers of the kernel will either restart the call
from the beginning or return the error to the user. If you return -ERESTARTSYS, you must
first undo any user-visible changes that might have been made, so that the right thing
happens when the system call is retried. If you cannot undo things in this manner, you
should return -EINTR instead.
scull_write must release the semaphore whether or not it was able to carry out its
other tasks successfully. If all goes well, execution falls into the final few lines of the
function:
out:
up(&dev->sem);
return retval;
This code frees the semaphore and returns whatever status is called for. There are
several places in scull_write where things can go wrong; these include memory allocation
failures or a fault while trying to copy data from user space. In those cases, the
code performs a goto out, ensuring that the proper cleanup is done.
Reader/Writer Semaphores
Semaphores perform mutual exclusion for all callers, regardless of what each thread
may want to do. Many tasks break down into two distinct types of work, however:
tasks that only need to read the protected data structures and those that must make
changes. It is often possible to allow multiple concurrent readers, as long as nobody
is trying to make any changes. Doing so can optimize performance significantly;
read-only tasks can get their work done in parallel without having to wait for other
readers to exit the critical section.
The Linux kernel provides a special type of semaphore called a rwsem (or "reader/writer
semaphore") for this situation. The use of rwsems in drivers is relatively rare, but they
are occasionally useful.
Code using rwsems must include <linux/rwsem.h>. The relevant data type for
reader/writer semaphores is struct rw_semaphore; an rwsem must be explicitly initialized
at runtime with:
void init_rwsem(struct rw_semaphore *sem);
A newly initialized rwsem is available for the next task(reader or writer) that comes
along. The interface for code needing read-only access is:
void down_read(struct rw_semaphore *sem);
int down_read_trylock(struct rw_semaphore *sem);
void up_read(struct rw_semaphore *sem);
A call to down_read provides read-only access to the protected resources, possibly
concurrently with other readers. Note that down_read may put the calling process
PAGE 114
into an uninterruptible sleep. down_read_trylock will not wait if read access is
unavailable; it returns nonzero if access was granted, 0 otherwise. Note that the convention
for down_read_trylock differs from that of most kernel functions, where success
is indicated by a return value of 0. A rwsem obtained with down_read must
eventually be freed with up_read.
The interface for writers is similar:
void down_write(struct rw_semaphore *sem);
int down_write_trylock(struct rw_semaphore *sem);
void up_write(struct rw_semaphore *sem);
void downgrade_write(struct rw_semaphore *sem);
down_write, down_write_trylock, and up_write all behave just like their reader counterparts,
except, of course, that they provide write access. If you have a situation
where a writer lockis needed for a quick change, followed by a longer period of readonly
access, you can use downgrade_write to allow other readers in once you have
finished making changes.
An rwsem allows either one writer or an unlimited number of readers to hold the
semaphore. Writers get priority; as soon as a writer tries to enter the critical section,
no readers will be allowed in until all writers have completed their work. This implementation
can lead to reader starvation--where readers are denied access for a long
time--if you have a large number of writers contending for the semaphore. For this
reason, rwsems are best used when write access is required only rarely, and writer
access is held for short periods of time.
Completions
A common pattern in kernel programming involves initiating some activity outside of
the current thread, then waiting for that activity to complete. This activity can be the
creation of a new kernel thread or user-space process, a request to an existing process,
or some sort of hardware-based action. It such cases, it can be tempting to use a
semaphore for synchronization of the two tasks, with code such as:
struct semaphore sem;
init_MUTEX_LOCKED(&sem);
start_external_task(&sem);
down(&sem);
The external task can then call up(&sem) when its work is done.
As is turns out, semaphores are not the best tool to use in this situation. In normal use,
code attempting to locka semaphore finds that semaphore available almost all the
time; if there is significant contention for the semaphore, performance suffers and the
locking scheme needs to be reviewed. So semaphores have been heavily optimized for
the "available" case. When used to communicate task completion in the way shown
above, however, the thread calling down will almost always have to wait; performance
PAGE 115
will suffer accordingly. Semaphores can also be subject to a (difficult) race condition
when used in this way if they are declared as automatic variables. In some cases, the
semaphore could vanish before the process calling up is finished with it.
These concerns inspired the addition of the "completion" interface in the 2.4.7 kernel.
Completions are a lightweight mechanism with one task: allowing one thread to
tell another that the job is done. To use completions, your code must include <linux/
completion.h>. A completion can be created with:
DECLARE_COMPLETION(my_completion);
Or, if the completion must be created and initialized dynamically:
struct completion my_completion;
/* ... */
init_completion(&my_completion);
Waiting for the completion is a simple matter of calling:
void wait_for_completion(struct completion *c);
Note that this function performs an uninterruptible wait. If your code calls wait_for_
completion and nobody ever completes the task, the result will be an unkillable
process.*
On the other side, the actual completion event may be signalled by calling one of the
following:
void complete(struct completion *c);
void complete_all(struct completion *c);
The two functions behave differently if more than one thread is waiting for the same
completion event. complete wakes up only one of the waiting threads while
complete_all allows all of them to proceed. In most cases, there is only one waiter,
and the two functions will produce an identical result.
A completion is normally a one-shot device; it is used once then discarded. It is possible,
however, to reuse completion structures if proper care is taken. If complete_all
is not used, a completion structure can be reused without any problems as long as
there is no ambiguity about what event is being signalled. If you use complete_all,
however, you must reinitialize the completion structure before reusing it. The macro:
INIT_COMPLETION(struct completion c);
can be used to quickly perform this reinitialization.
As an example of how completions may be used, consider the complete module, which
is included in the example source. This module defines a device with simple semantics:
any process trying to read from the device will wait (using wait_for_completion)
* As of this writing, patches adding interruptible versions were in circulation but had not been merged into the mainline.
PAGE 116
until some other process writes to the device. The code which implements this behavior
is:
DECLARE_COMPLETION(comp);
ssize_t complete_read (struct file *filp, char __user *buf, size_t count, loff_t
*pos)
{
printk(KERN_DEBUG "process %i (%s) going to sleep\n",
current->pid, current->comm);
wait_for_completion(&comp);
printk(KERN_DEBUG "awoken %i (%s)\n", current->pid, current->comm);
return 0; /* EOF */
}
ssize_t complete_write (struct file *filp, const char __user *buf, size_t count,
loff_t *pos)
{
printk(KERN_DEBUG "process %i (%s) awakening the readers...\n",
current->pid, current->comm);
complete(&comp);
return count; /* succeed, to avoid retrial */
}
It is possible to have multiple processes "reading" from this device at the same time.
Each write to the device will cause exactly one read operation to complete, but there
is no way to know which one it will be.
A typical use of the completion mechanism is with kernel thread termination at module
exit time. In the prototypical case, some of the driver internal workings is performed
by a kernel thread in a while (1) loop. When the module is ready to be
cleaned up, the exit function tells the thread to exit and then waits for completion.
To this aim, the kernel includes a specific function to be used by the thread:
void complete_and_exit(struct completion *c, long retval);
Spinlocks
Semaphores are a useful tool for mutual exclusion, but they are not the only such
tool provided by the kernel. Instead, most locking is implemented with a mechanism
called a spinlock. Unlike semaphores, spinlocks may be used in code that cannot
sleep, such as interrupt handlers. When properly used, spinlocks offer higher
performance than semaphores in general. They do, however, bring a different set of
constraints on their use.
Spinlocks are simple in concept. A spinlock is a mutual exclusion device that can
have only two values: "locked" and "unlocked." It is usually implemented as a single
bit in an integer value. Code wishing to take out a particular lock tests the relevant
bit. If the lockis available, the "locked" bit is set and the code continues into the critical
section. If, instead, the lockhas been taken by somebody else, the code goes into
PAGE 117
a tight loop where it repeatedly checks the lock until it becomes available. This loop
is the "spin" part of a spinlock.
Of course, the real implementation of a spinlock is a bit more complex than the
description above. The "test and set" operation must be done in an atomic manner
so that only one thread can obtain the lock, even if several are spinning at any given
time. Care must also be taken to avoid deadlocks on hyperthreaded processors-chips
that implement multiple, virtual CPUs sharing a single processor core and
cache. So the actual spinlock implementation is different for every architecture that
Linux supports. The core concept is the same on all systems, however, when there is
contention for a spinlock, the processors that are waiting execute a tight loop and
accomplish no useful work.
Spinlocks are, by their nature, intended for use on multiprocessor systems, although
a uniprocessor workstation running a preemptive kernel behaves like SMP, as far as
concurrency is concerned. If a nonpreemptive uniprocessor system ever went into a
spin on a lock, it would spin forever; no other thread would ever be able to obtain
the CPU to release the lock. For this reason, spinlock operations on uniprocessor systems
without preemption enabled are optimized to do nothing, with the exception of
the ones that change the IRQ masking status. Because of preemption, even if you
never expect your code to run on an SMP system, you still need to implement proper
locking.
Introduction to the Spinlock API
The required include file for the spinlock primitives is <linux/spinlock.h>. An actual
lockhas the type spinlock_t. Like any other data structure, a spinlock must be initialized.
This initialization may be done at compile time as follows:
spinlock_t my_lock = SPIN_LOCK_UNLOCKED;
or at runtime with:
void spin_lock_init(spinlock_t *lock);
Before entering a critical section, your code must obtain the requisite lock with:
void spin_lock(spinlock_t *lock);
Note that all spinlock waits are, by their nature, uninterruptible. Once you call
spin_lock, you will spin until the lock becomes available.
To release a lock that you have obtained, pass it to:
void spin_unlock(spinlock_t *lock);
There are many other spinlock-functions, and we will look at them all shortly. But
none of them depart from the core idea shown by the functions listed above. There is
very little that one can do with a lock, other than lock and release it. However, there
PAGE 118
are a few rules about how you must work with spinlocks. We will take a moment to
look at those before getting into the full spinlock interface.
Spinlocks and Atomic Context
Imagine for a moment that your driver acquires a spinlock and goes about its business
within its critical section. Somewhere in the middle, your driver loses the processor.
Perhaps it has called a function (copy_from_user, say) that puts the process to
sleep. Or, perhaps, kernel preemption kicks in, and a higher-priority process pushes
your code aside. Your code is now holding a lockthat it will not release any time in
the foreseeable future. If some other thread tries to obtain the same lock, it will, in
the best case, wait (spinning in the processor) for a very long time. In the worst case,
the system could deadlock entirely.
Most readers would agree that this scenario is best avoided. Therefore, the core rule
that applies to spinlocks is that any code must, while holding a spinlock, be atomic.
It cannot sleep; in fact, it cannot relinquish the processor for any reason except to
service interrupts (and sometimes not even then).
The kernel preemption case is handled by the spinlock code itself. Any time kernel
code holds a spinlock, preemption is disabled on the relevant processor. Even uniprocessor
systems must disable preemption in this way to avoid race conditions.
That is why proper locking is required even if you never expect your code to run on a
multiprocessor machine.
Avoiding sleep while holding a lockcan be more difficult; many kernel functions can
sleep, and this behavior is not always well documented. Copying data to or from user
space is an obvious example: the required user-space page may need to be swapped
in from the disk before the copy can proceed, and that operation clearly requires a
sleep. Just about any operation that must allocate memory can sleep; kmalloc can
decide to give up the processor, and wait for more memory to become available
unless it is explicitly told not to. Sleeps can happen in surprising places; writing code
that will execute under a spinlock requires paying attention to every function that
you call.
Here's another scenario: your driver is executing and has just taken out a lock that
controls access to its device. While the lockis held, the device issues an interrupt,
which causes your interrupt handler to run. The interrupt handler, before accessing
the device, must also obtain the lock. Taking out a spinlock in an interrupt handler is
a legitimate thing to do; that is one of the reasons that spinlock operations do not
sleep. But what happens if the interrupt routine executes in the same processor as the
code that took out the lock originally? While the interrupt handler is spinning, the
noninterrupt code will not be able to run to release the lock. That processor will spin
forever.
PAGE 119
Avoiding this trap requires disabling interrupts (on the local CPU only) while the
spinlock is held. There are variants of the spinlock functions that will disable interrupts
for you (we'll see them in the next section). However, a complete discussion of
interrupts must wait until Chapter 10.
The last important rule for spinlock usage is that spinlocks must always be held for
the minimum time possible. The longer you hold a lock, the longer another processor
may have to spin waiting for you to release it, and the chance of it having to spin
at all is greater. Long lockhold times also keep the current processor from scheduling,
meaning that a higher priority process--which really should be able to get the
CPU--may have to wait. The kernel developers put a great deal of effort into reducing
kernel latency (the time a process may have to wait to be scheduled) in the 2.5
development series. A poorly written driver can wipe out all that progress just by
holding a lockfor too long. To avoid creating this sort of problem, make a point of
keeping your lock-hold times short.
The Spinlock Functions
We have already seen two functions, spin_lock and spin_unlock, that manipulate spinlocks.
There are several other functions, however, with similar names and purposes.
We will now present the full set. This discussion will take us into ground we will not
be able to cover properly for a few chapters yet; a complete understanding of the spinlock
API requires an understanding of interrupt handling and related concepts.
There are actually four functions that can lock a spinlock:
void spin_lock(spinlock_t *lock);
void spin_lock_irqsave(spinlock_t *lock, unsigned long flags);
void spin_lock_irq(spinlock_t *lock);
void spin_lock_bh(spinlock_t *lock)
We have already seen how spin_lock works. spin_lock_irqsave disables interrupts (on
the local processor only) before taking the spinlock; the previous interrupt state is
stored in flags. If you are absolutely sure nothing else might have already disabled
interrupts on your processor (or, in other words, you are sure that you should enable
interrupts when you release your spinlock), you can use spin_lock_irq instead and
not have to keep track of the flags. Finally, spin_lock_bh disables software interrupts
before taking the lock, but leaves hardware interrupts enabled.
If you have a spinlock that can be taken by code that runs in (hardware or software)
interrupt context, you must use one of the forms of spin_lock that disables interrupts.
Doing otherwise can deadlock the system, sooner or later. If you do not access
your lockin a hardware interrupt handler, but you do via software interrupts (in
code that runs out of a tasklet, for example, a topic covered in Chapter 7), you can
use spin_lock_bh to safely avoid deadlocks while still allowing hardware interrupts to
be serviced.
PAGE 120
There are also four ways to release a spinlock; the one you use must correspond to
the function you used to take the lock:
void spin_unlock(spinlock_t *lock);
void spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags);
void spin_unlock_irq(spinlock_t *lock);
void spin_unlock_bh(spinlock_t *lock);
Each spin_unlock variant undoes the work performed by the corresponding spin_lock
function. The flags argument passed to spin_unlock_irqrestore must be the same
variable passed to spin_lock_irqsave. You must also call spin_lock_irqsave and spin_
unlock_irqrestore in the same function; otherwise, your code may break on some
architectures.
There is also a set of nonblocking spinlock operations:
int spin_trylock(spinlock_t *lock);
int spin_trylock_bh(spinlock_t *lock);
These functions return nonzero on success (the lock was obtained), 0 otherwise.
There is no "try" version that disables interrupts.
Reader/Writer Spinlocks
The kernel provides a reader/writer form of spinlocks that is directly analogous to
the reader/writer semaphores we saw earlier in this chapter. These locks allow any
number of readers into a critical section simultaneously, but writers must have exclusive
access. Reader/writer locks have a type of rwlock_t, defined in <linux/spinlock.h>.
They can be declared and initialized in two ways:
rwlock_t my_rwlock = RW_LOCK_UNLOCKED; /* Static way */
rwlock_t my_rwlock;
rwlock_init(&my_rwlock); /* Dynamic way */
The list of functions available should look reasonably familiar by now. For readers,
the following functions are available:
void read_lock(rwlock_t *lock);
void read_lock_irqsave(rwlock_t *lock, unsigned long flags);
void read_lock_irq(rwlock_t *lock);
void read_lock_bh(rwlock_t *lock);
void read_unlock(rwlock_t *lock);
void read_unlock_irqrestore(rwlock_t *lock, unsigned long flags);
void read_unlock_irq(rwlock_t *lock);
void read_unlock_bh(rwlock_t *lock);
Interestingly, there is no read_trylock.
The functions for write access are similar:
void write_lock(rwlock_t *lock);
void write_lock_irqsave(rwlock_t *lock, unsigned long flags);
PAGE 121
void write_lock_irq(rwlock_t *lock);
void write_lock_bh(rwlock_t *lock);
int write_trylock(rwlock_t *lock);
void write_unlock(rwlock_t *lock);
void write_unlock_irqrestore(rwlock_t *lock, unsigned long flags);
void write_unlock_irq(rwlock_t *lock);
void write_unlock_bh(rwlock_t *lock);
Reader/writer locks can starve readers just as rwsems can. This behavior is rarely a
problem; however, if there is enough lockcontention to bring about starvation, performance
is poor anyway.
Locking Traps
Many years of experience with locks--experience that predates Linux--have shown
that locking can be very hard to get right. Managing concurrency is an inherently
tricky undertaking, and there are many ways of making mistakes. In this section, we
take a quick look at things that can go wrong.
Ambiguous Rules
As has already been said above, a proper locking scheme requires clear and explicit
rules. When you create a resource that can be accessed concurrently, you should
define which lockwill control that access. Locking should really be laid out at the
beginning; it can be a hard thing to retrofit in afterward. Time taken at the outset
usually is paid back generously at debugging time.
As you write your code, you will doubtless encounter several functions that all
require access to structures protected by a specific lock. At this point, you must be
careful: if one function acquires a lockand then calls another function that also
attempts to acquire the lock, your code deadlocks. Neither semaphores nor spinlocks
allow a lockholder to acquire the locka second time; should you attempt to do
so, things simply hang.
To make your locking work properly, you have to write some functions with the
assumption that their caller has already acquired the relevant lock(s). Usually, only
your internal, static functions can be written in this way; functions called from outside
must handle locking explicitly. When you write internal functions that make
assumptions about locking, do yourself (and anybody else who works with your
code) a favor and document those assumptions explicitly. It can be very hard to
come back months later and figure out whether you need to hold a lockto call a particular
function or not.
In the case of scull, the design decision taken was to require all functions invoked
directly from system calls to acquire the semaphore applying to the device structure
PAGE 122
that is accessed. All internal functions, which are only called from other scull functions,
can then assume that the semaphore has been properly acquired.
Lock Ordering Rules
In systems with a large number of locks (and the kernel is becoming such a system),
it is not unusual for code to need to hold more than one lockat once. If some sort of
computation must be performed using two different resources, each of which has its
own lock, there is often no alternative to acquiring both locks.
Taking multiple locks can be dangerous, however. If you have two locks, called
Lock1 and Lock2, and code needs to acquire both at the same time, you have a
potential deadlock. Just imagine one thread locking Lock1 while another simultaneously
takes Lock2. Then each thread tries to get the one it doesn't have. Both
threads will deadlock.
The solution to this problem is usually simple: when multiple locks must be
acquired, they should always be acquired in the same order. As long as this convention
is followed, simple deadlocks like the one described above can be avoided.
However, following lockordering rules can be easier said than done. It is very rare
that such rules are actually written down anywhere. Often the best you can do is to
see what other code does.
A couple of rules of thumb can help. If you must obtain a lockthat is local to your
code (a device lock, say) along with a lock belonging to a more central part of the
kernel, take your lock first. If you have a combination of semaphores and spinlocks,
you must, of course, obtain the semaphore(s) first; calling down (which can sleep)
while holding a spinlock is a serious error. But most of all, try to avoid situations
where you need more than one lock.
Fine- Versus Coarse-Grained Locking
The first Linux kernel that supported multiprocessor systems was 2.0; it contained
exactly one spinlock. The big kernel lock turned the entire kernel into one large critical
section; only one CPU could be executing kernel code at any given time. This
locksolved the concurrency problem well enough to allow the kernel developers to
address all of the other issues involved in supporting SMP. But it did not scale very
well. Even a two-processor system could spend a significant amount of time simply
waiting for the big kernel lock. The performance of a four-processor system was not
even close to that of four independent machines.
So, subsequent kernel releases have included finer-grained locking. In 2.2, one spinlock
controlled access to the block I/O subsystem; another worked for networking,
and so on. A modern kernel can contain thousands of locks, each protecting one
small resource. This sort of fine-grained locking can be good for scalability; it allows
PAGE 123
each processor to work on its specific task without contending for locks used by
other processors. Very few people miss the big kernel lock.*
Fine-grained locking comes at a cost, however. In a kernel with thousands of locks, it
can be very hard to know which locks you need--and in which order you should
acquire them--to perform a specific operation. Remember that locking bugs can be
very difficult to find; more locks provide more opportunities for truly nasty locking
bugs to creep into the kernel. Fine-grained locking can bring a level of complexity
that, over the long term, can have a large, adverse effect on the maintainability of the
kernel.
Locking in a device driver is usually relatively straightforward; you can have a single
lockthat covers everything you do, or you can create one lockfor every device you
manage. As a general rule, you should start with relatively coarse locking unless you
have a real reason to believe that contention could be a problem. Resist the urge to
optimize prematurely; the real performance constraints often show up in unexpected
places.
If you do suspect that lockcontention is hurting performance, you may find the lockmeter
tool useful. This patch (available at http://oss.sgi.com/projects/lockmeter/)
instruments the kernel to measure time spent waiting in locks. By looking at the
report, you are able to determine quickly whether lock contention is truly the problem
or not.
Alternatives to Locking
The Linux kernel provides a number of powerful locking primitives that can be used
to keep the kernel from tripping over its own feet. But, as we have seen, the design
and implementation of a locking scheme is not without its pitfalls. Often there is no
alternative to semaphores and spinlocks; they may be the only way to get the job
done properly. There are situations, however, where atomic access can be set up
without the need for full locking. This section looks at other ways of doing things.
Lock-Free Algorithms
Sometimes, you can recast your algorithms to avoid the need for locking altogether.
A number of reader/writer situations--if there is only one writer--can often workin
this manner. If the writer takes care that the view of the data structure, as seen by the
reader, is always consistent, it may be possible to create a lock-free data structure.
A data structure that can often be useful for lockless producer/consumer tasks is the
circular buffer. This algorithm involves a producer placing data into one end of an
* This lockstill exists in 2.6, though it covers very little of the kernel now. If you stumble
across a lock_kernel call, you have found the big kernel lock. Do not even think about
using it in any new code, however.
PAGE 124
array, while the consumer removes data from the other. When the end of the array is
reached, the producer wraps back around to the beginning. So a circular buffer
requires an array and two index values to track where the next new value goes and
which value should be removed from the buffer next.
When carefully implemented, a circular buffer requires no locking in the absence of
multiple producers or consumers. The producer is the only thread that is allowed to
modify the write index and the array location it points to. As long as the writer stores
a new value into the buffer before updating the write index, the reader will always
see a consistent view. The reader, in turn, is the only thread that can access the read
index and the value it points to. With a bit of care to ensure that the two pointers do
not overrun each other, the producer and the consumer can access the buffer concurrently
with no race conditions.
Figure 5-1 shows circular buffer in several states of fill. This buffer has been defined
such that an empty condition is indicated by the read and write pointers being equal,
while a full condition happens whenever the write pointer is immediately behind the
read pointer (being careful to account for a wrap!). When carefully programmed, this
buffer can be used without locks.
Figure 5-1. A circular buffer
Circular buffers show up reasonably often in device drivers. Networking adaptors, in
particular, often use circular buffers to exchange data (packets) with the processor.
Note that, as of 2.6.10, there is a generic circular buffer implementation available in
the kernel; see <linux/kfifo.h> for information on how to use it.
Atomic Variables
Sometimes, a shared resource is a simple integer value. Suppose your driver maintains
a shared variable n_op that tells how many device operations are currently outstanding.
Normally, even a simple operation such as:
n_op++;
PAGE 125
would require locking. Some processors might perform that sort of increment in an
atomic manner, but you can't count on it. But a full locking regime seems like overhead
for a simple integer value. For cases like this, the kernel provides an atomic
integer type called atomic_t, defined in <asm/atomic.h>.
An atomic_t holds an int value on all supported architectures. Because of the way
this type works on some processors, however, the full integer range may not be available;
thus, you should not count on an atomic_t holding more than 24 bits. The following
operations are defined for the type and are guaranteed to be atomic with
respect to all processors of an SMP computer. The operations are very fast, because
they compile to a single machine instruction whenever possible.
void atomic_set(atomic_t *v, int i);
atomic_t v = ATOMIC_INIT(0);
Set the atomic variable v to the integer value i. You can also initialize atomic values
at compile time with the ATOMIC_INIT macro.
int atomic_read(atomic_t *v);
Return the current value of v.
void atomic_add(int i, atomic_t *v);
Add i to the atomic variable pointed to by v. The return value is void, because
there is an extra cost to returning the new value, and most of the time there's no
need to know it.
void atomic_sub(int i, atomic_t *v);
Subtract i from *v.
void atomic_inc(atomic_t *v);
void atomic_dec(atomic_t *v);
Increment or decrement an atomic variable.
int atomic_inc_and_test(atomic_t *v);
int atomic_dec_and_test(atomic_t *v);
int atomic_sub_and_test(int i, atomic_t *v);
Perform the specified operation and test the result; if, after the operation, the
atomic value is 0, then the return value is true; otherwise, it is false. Note that
there is no atomic_add_and_test.
int atomic_add_negative(int i, atomic_t *v);
Add the integer variable i to v. The return value is true if the result is negative,
false otherwise.
int atomic_add_return(int i, atomic_t *v);
int atomic_sub_return(int i, atomic_t *v);
int atomic_inc_return(atomic_t *v);
int atomic_dec_return(atomic_t *v);
Behave just like atomic_add and friends, with the exception that they return the
new value of the atomic variable to the caller.
PAGE 126
As stated earlier, atomic_t data items must be accessed only through these functions.
If you pass an atomic item to a function that expects an integer argument, you'll get
a compiler error.
You should also bear in mind that atomic_t values work only when the quantity in
question is truly atomic. Operations requiring multiple atomic_t variables still
require some other sort of locking. Consider the following code:
atomic_sub(amount, &first_atomic);
atomic_add(amount, &second_atomic);
There is a period of time where the amount has been subtracted from the first atomic
value but not yet added to the second. If that state of affairs could create trouble for
code that might run between the two operations, some form of locking must be
employed.
Bit Operations
The atomic_t type is good for performing integer arithmetic. It doesn't work as well,
however, when you need to manipulate individual bits in an atomic manner. For that
purpose, instead, the kernel offers a set of functions that modify or test single bits
atomically. Because the whole operation happens in a single step, no interrupt (or
other processor) can interfere.
Atomic bit operations are very fast, since they perform the operation using a single
machine instruction without disabling interrupts whenever the underlying platform
can do that. The functions are architecture dependent and are declared in <asm/
bitops.h>. They are guaranteed to be atomic even on SMP computers and are useful
to keep coherence across processors.
Unfortunately, data typing in these functions is architecture dependent as well. The
nr argument (describing which bit to manipulate) is usually defined as int but is
unsigned long for a few architectures. The address to be modified is usually a pointer
to unsigned long, but a few architectures use void * instead.
The available bit operations are:
void set_bit(nr, void *addr);
Sets bit number nr in the data item pointed to by addr.
void clear_bit(nr, void *addr);
Clears the specified bit in the unsigned long datum that lives at addr. Its semantics
are otherwise the same as set_bit.
void change_bit(nr, void *addr);
Toggles the bit.
PAGE 127
test_bit(nr, void *addr);
This function is the only bit operation that doesn't need to be atomic; it simply
returns the current value of the bit.
int test_and_set_bit(nr, void *addr);
int test_and_clear_bit(nr, void *addr);
int test_and_change_bit(nr, void *addr);
Behave atomically like those listed previously, except that they also return the
previous value of the bit.
When these functions are used to access and modify a shared flag, you don't have to
do anything except call them; they perform their operations in an atomic manner.
Using bit operations to manage a lockvariable that controls access to a shared variable,
on the other hand, is a little more complicated and deserves an example. Most
modern code does not use bit operations in this way, but code like the following still
exists in the kernel.
A code segment that needs to access a shared data item tries to atomically acquire a
lockusing either test_and_set_bit or test_and_clear_bit. The usual implementation is
shown here; it assumes that the locklives at bit nr of address addr. It also assumes
that the bit is 0 when the lock is free or nonzero when the lock is busy.
/* try to set lock */
while (test_and_set_bit(nr, addr) != 0)
wait_for_a_while( );
/* do your work */
/* release lock, and check... */
if (test_and_clear_bit(nr, addr) = = 0)
something_went_wrong( ); /* already released: error */
If you read through the kernel source, you find code that works like this example. It
is, however, far better to use spinlocks in new code; spinlocks are well debugged,
they handle issues like interrupts and kernel preemption, and others reading your
code do not have to work to understand what you are doing.
seqlocks
The 2.6 kernel contains a couple of new mechanisms that are intended to provide
fast, lockless access to a shared resource. Seqlocks work in situations where the
resource to be protected is small, simple, and frequently accessed, and where write
access is rare but must be fast. Essentially, they work by allowing readers free access
to the resource but requiring those readers to check for collisions with writers and,
when such a collision happens, retry their access. Seqlocks generally cannot be used
to protect data structures involving pointers, because the reader may be following a
pointer that is invalid while the writer is changing the data structure.
PAGE 128
Seqlocks are defined in <linux/seqlock.h>. There are the two usual methods for initializing
a seqlock (which has type seqlock_t):
seqlock_t lock1 = SEQLOCK_UNLOCKED;
seqlock_t lock2;
seqlock_init(&lock2);
Read access works by obtaining an (unsigned) integer sequence value on entry into
the critical section. On exit, that sequence value is compared with the current value;
if there is a mismatch, the read access must be retried. As a result, reader code has a
form like the following:
unsigned int seq;
do {
seq = read_seqbegin(&the_lock);
/* Do what you need to do */
} while read_seqretry(&the_lock, seq);
This sort of lockis usually used to protect some sort of simple computation that
requires multiple, consistent values. If the test at the end of the computation shows
that a concurrent write occurred, the results can be simply discarded and recomputed.
If your seqlock might be accessed from an interrupt handler, you should use the
IRQ-safe versions instead:
unsigned int read_seqbegin_irqsave(seqlock_t *lock,
unsigned long flags);
int read_seqretry_irqrestore(seqlock_t *lock, unsigned int seq,
unsigned long flags);
Writers must obtain an exclusive lockto enter the critical section protected by a
seqlock. To do so, call:
void write_seqlock(seqlock_t *lock);
The write lockis implemented with a spinlock, so all the usual constraints apply.
Make a call to:
void write_sequnlock(seqlock_t *lock);
to release the lock. Since spinlocks are used to control write access, all of the usual
variants are available:
void write_seqlock_irqsave(seqlock_t *lock, unsigned long flags);
void write_seqlock_irq(seqlock_t *lock);
void write_seqlock_bh(seqlock_t *lock);
void write_sequnlock_irqrestore(seqlock_t *lock, unsigned long flags);
void write_sequnlock_irq(seqlock_t *lock);
void write_sequnlock_bh(seqlock_t *lock);
There is also a write_tryseqlock that returns nonzero if it was able to obtain the lock.
PAGE 129
Read-Copy-Update
Read-copy-update (RCU) is an advanced mutual exclusion scheme that can yield
high performance in the right conditions. Its use in drivers is rare but not unknown,
so it is worth a quick overview here. Those who are interested in the full details of
the RCU algorithm can find them in the white paper published by its creator
(http://www.rdrop.com/users/paulmck/rclock/intro/rclock_intro.html).
RCU places a number of constraints on the sort of data structure that it can protect.
It is optimized for situations where reads are common and writes are rare. The
resources being protected should be accessed via pointers, and all references to those
resources must be held only by atomic code. When the data structure needs to be
changed, the writing thread makes a copy, changes the copy, then aims the relevant
pointer at the new version--thus, the name of the algorithm. When the kernel is sure
that no references to the old version remain, it can be freed.
As an example of real-world use of RCU, consider the network routing tables. Every
outgoing packet requires a check of the routing tables to determine which interface
should be used. The check is fast, and, once the kernel has found the target interface,
it no longer needs the routing table entry. RCU allows route lookups to be performed
without locking, with significant performance benefits. The Starmode radio
IP driver in the kernel also uses RCU to keep track of its list of devices.
Code using RCU should include <linux/rcupdate.h>.
On the read side, code using an RCU-protected data structure should bracket its references
with calls to rcu_read_lock and rcu_read_unlock. As a result, RCU code
tends to look like:
struct my_stuff *stuff;
rcu_read_lock( );
stuff = find_the_stuff(args...);
do_something_with(stuff);
rcu_read_unlock( );
The rcu_read_lock call is fast; it disables kernel preemption but does not wait for
anything. The code that executes while the read "lock" is held must be atomic. No
reference to the protected resource may be used after the call to rcu_read_unlock.
Code that needs to change the protected structure has to carry out a few steps. The
first part is easy; it allocates a new structure, copies data from the old one if need be,
then replaces the pointer that is seen by the read code. At this point, for the purposes
of the read side, the change is complete; any code entering the critical section
sees the new version of the data.
All that remains is to free the old version. The problem, of course, is that code running
on other processors may still have a reference to the older data, so it cannot be freed
immediately. Instead, the write code must wait until it knows that no such reference
PAGE 130
can exist. Since all code holding references to this data structure must (by the rules) be
atomic, we know that once every processor on the system has been scheduled at least
once, all references must be gone. So that is what RCU does; it sets aside a callback
that waits until all processors have scheduled; that callback is then run to perform the
cleanup work.
Code that changes an RCU-protected data structure must get its cleanup callback by
allocating a struct rcu_head, although it doesn't need to initialize that structure in
any way. Often, that structure is simply embedded within the larger resource that is
protected by RCU. After the change to that resource is complete, a call should be
made to:
void call_rcu(struct rcu_head *head, void (*func)(void *arg), void *arg);
The given func is called when it is safe to free the resource; it is passed to the same
arg that was passed to call_rcu. Usually, the only thing func needs to do is to call
kfree.
The full RCU interface is more complex than we have seen here; it includes, for
example, utility functions for working with protected linked lists. See the relevant
header files for the full story.
Quick Reference
This chapter has introduced a substantial set of symbols for the management of concurrency.
The most important of these are summarized here:
#include <asm/semaphore.h>
The include file that defines semaphores and the operations on them.
DECLARE_MUTEX(name);
DECLARE_MUTEX_LOCKED(name);
Two macros for declaring and initializing a semaphore used in mutual exclusion
mode.
void init_MUTEX(struct semaphore *sem);
void init_MUTEX_LOCKED(struct semaphore *sem);
These two functions can be used to initialize a semaphore at runtime.
void down(struct semaphore *sem);
int down_interruptible(struct semaphore *sem);
int down_trylock(struct semaphore *sem);
void up(struct semaphore *sem);
Lock and unlock a semaphore. down puts the calling process into an uninterruptible
sleep if need be; down_interruptible, instead, can be interrupted by a signal.
down_trylock does not sleep; instead, it returns immediately if the
semaphore is unavailable. Code that locks a semaphore must eventually unlock
it with up.
PAGE 131
struct rw_semaphore;
init_rwsem(struct rw_semaphore *sem);
The reader/writer version of semaphores and the function that initializes it.
void down_read(struct rw_semaphore *sem);
int down_read_trylock(struct rw_semaphore *sem);
void up_read(struct rw_semaphore *sem);
Functions for obtaining and releasing read access to a reader/writer semaphore.
void down_write(struct rw_semaphore *sem)
int down_write_trylock(struct rw_semaphore *sem)
void up_write(struct rw_semaphore *sem)
void downgrade_write(struct rw_semaphore *sem)
Functions for managing write access to a reader/writer semaphore.
#include <linux/completion.h>
DECLARE_COMPLETION(name);
init_completion(struct completion *c);
INIT_COMPLETION(struct completion c);
The include file describing the Linux completion mechanism, and the normal
methods for initializing completions. INIT_COMPLETION should be used only to
reinitialize a completion that has been previously used.
void wait_for_completion(struct completion *c);
Wait for a completion event to be signalled.
void complete(struct completion *c);
void complete_all(struct completion *c);
Signal a completion event. complete wakes, at most, one waiting thread, while
complete_all wakes all waiters.
void complete_and_exit(struct completion *c, long retval);
Signals a completion event by calling complete and calls exit for the current
thread.
#include <linux/spinlock.h>
spinlock_t lock = SPIN_LOCK_UNLOCKED;
spin_lock_init(spinlock_t *lock);
The include file defining the spinlock interface and the two ways of initializing
locks.
void spin_lock(spinlock_t *lock);
void spin_lock_irqsave(spinlock_t *lock, unsigned long flags);
void spin_lock_irq(spinlock_t *lock);
void spin_lock_bh(spinlock_t *lock);
The various ways of locking a spinlock and, possibly, disabling interrupts.
PAGE 132
int spin_trylock(spinlock_t *lock);
int spin_trylock_bh(spinlock_t *lock);
Nonspinning versions of the above functions; these return 0 in case of failure to
obtain the lock, nonzero otherwise.
void spin_unlock(spinlock_t *lock);
void spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags);
void spin_unlock_irq(spinlock_t *lock);
void spin_unlock_bh(spinlock_t *lock);
The corresponding ways of releasing a spinlock.
rwlock_t lock = RW_LOCK_UNLOCKED
rwlock_init(rwlock_t *lock);
The two ways of initializing reader/writer locks.
void read_lock(rwlock_t *lock);
void read_lock_irqsave(rwlock_t *lock, unsigned long flags);
void read_lock_irq(rwlock_t *lock);
void read_lock_bh(rwlock_t *lock);
Functions for obtaining read access to a reader/writer lock.
void read_unlock(rwlock_t *lock);
void read_unlock_irqrestore(rwlock_t *lock, unsigned long flags);
void read_unlock_irq(rwlock_t *lock);
void read_unlock_bh(rwlock_t *lock);
Functions for releasing read access to a reader/writer spinlock.
void write_lock(rwlock_t *lock);
void write_lock_irqsave(rwlock_t *lock, unsigned long flags);
void write_lock_irq(rwlock_t *lock);
void write_lock_bh(rwlock_t *lock);
Functions for obtaining write access to a reader/writer lock.
void write_unlock(rwlock_t *lock);
void write_unlock_irqrestore(rwlock_t *lock, unsigned long flags);
void write_unlock_irq(rwlock_t *lock);
void write_unlock_bh(rwlock_t *lock);
Functions for releasing write access to a reader/writer spinlock.
PAGE 133
#include <asm/atomic.h>
atomic_t v = ATOMIC_INIT(value);
void atomic_set(atomic_t *v, int i);
int atomic_read(atomic_t *v);
void atomic_add(int i, atomic_t *v);
void atomic_sub(int i, atomic_t *v);
void atomic_inc(atomic_t *v);
void atomic_dec(atomic_t *v);
int atomic_inc_and_test(atomic_t *v);
int atomic_dec_and_test(atomic_t *v);
int atomic_sub_and_test(int i, atomic_t *v);
int atomic_add_negative(int i, atomic_t *v);
int atomic_add_return(int i, atomic_t *v);
int atomic_sub_return(int i, atomic_t *v);
int atomic_inc_return(atomic_t *v);
int atomic_dec_return(atomic_t *v);
Atomically access integer variables. The atomic_t variables must be accessed
only through these functions.
#include <asm/bitops.h>
void set_bit(nr, void *addr);
void clear_bit(nr, void *addr);
void change_bit(nr, void *addr);
test_bit(nr, void *addr);
int test_and_set_bit(nr, void *addr);
int test_and_clear_bit(nr, void *addr);
int test_and_change_bit(nr, void *addr);
Atomically access bit values; they can be used for flags or lockvariables. Using
these functions prevents any race condition related to concurrent access to the
bit.
#include <linux/seqlock.h>
seqlock_t lock = SEQLOCK_UNLOCKED;
seqlock_init(seqlock_t *lock);
The include file defining seqlocks and the two ways of initializing them.
unsigned int read_seqbegin(seqlock_t *lock);
unsigned int read_seqbegin_irqsave(seqlock_t *lock, unsigned long flags);
int read_seqretry(seqlock_t *lock, unsigned int seq);
int read_seqretry_irqrestore(seqlock_t *lock, unsigned int seq, unsigned long flags);
Functions for obtaining read access to a seqlock-protected resources.
PAGE 134
void write_seqlock(seqlock_t *lock);
void write_seqlock_irqsave(seqlock_t *lock, unsigned long flags);
void write_seqlock_irq(seqlock_t *lock);
void write_seqlock_bh(seqlock_t *lock);
int write_tryseqlock(seqlock_t *lock);
Functions for obtaining write access to a seqlock-protected resource.
void write_sequnlock(seqlock_t *lock);
void write_sequnlock_irqrestore(seqlock_t *lock, unsigned long flags);
void write_sequnlock_irq(seqlock_t *lock);
void write_sequnlock_bh(seqlock_t *lock);
Functions for releasing write access to a seqlock-protected resource.
#include <linux/rcupdate.h>
The include file required to use the read-copy-update (RCU) mechanism.
void rcu_read_lock;
void rcu_read_unlock;
Macros for obtaining atomic read access to a resource protected by RCU.
void call_rcu(struct rcu_head *head, void (*func)(void *arg), void *arg);
Arranges for a callback to run after all processors have been scheduled and an
RCU-protected resource can be safely freed.
PAGE 135
CHAPTER 6
Advanced Char Driver Operations
In Chapter 3, we built a complete device driver that the user can write to and read
from. But a real device usually offers more functionality than synchronous read and
write. Now that we're equipped with debugging tools should something go awry-and
a firm understanding of concurrency issues to help keep things from going
awry--we can safely go ahead and create a more advanced driver.
This chapter examines a few concepts that you need to understand to write fully featured
char device drivers. We start with implementing the ioctl system call, which is
a common interface used for device control. Then we proceed to various ways of synchronizing
with user space; by the end of this chapter you have a good idea of how to
put processes to sleep (and wake them up), implement nonblocking I/O, and inform
user space when your devices are available for reading or writing. We finish with a
look at how to implement a few different device access policies within drivers.
The ideas discussed here are demonstrated by way of a couple of modified versions
of the scull driver. Once again, everything is implemented using in-memory virtual
devices, so you can try out the code yourself without needing to have any particular
hardware. By now, you may be wanting to get your hands dirty with real hardware,
but that will have to wait until Chapter 9.
ioctl
Most drivers need--in addition to the ability to read and write the device--the ability
to perform various types of hardware control via the device driver. Most devices
can perform operations beyond simple data transfers; user space must often be able
to request, for example, that the device lock its door, eject its media, report error
information, change a baud rate, or self destruct. These operations are usually supported
via the ioctl method, which implements the system call by the same name.
In user space, the ioctl system call has the following prototype:
int ioctl(int fd, unsigned long cmd, ...);
PAGE 136
The prototype stands out in the list of Unix system calls because of the dots, which
usually mark the function as having a variable number of arguments. In a real system,
however, a system call can't actually have a variable number of arguments. System
calls must have a well-defined prototype, because user programs can access
them only through hardware "gates." Therefore, the dots in the prototype represent
not a variable number of arguments but a single optional argument, traditionally
identified as char *argp. The dots are simply there to prevent type checking during
compilation. The actual nature of the third argument depends on the specific control
command being issued (the second argument). Some commands take no arguments,
some take an integer value, and some take a pointer to other data. Using a
pointer is the way to pass arbitrary data to the ioctl call; the device is then able to
exchange any amount of data with user space.
The unstructured nature of the ioctl call has caused it to fall out of favor among kernel
developers. Each ioctl command is, essentially, a separate, usually undocumented
system call, and there is no way to audit these calls in any sort of
comprehensive manner. It is also difficult to make the unstructured ioctl arguments
work identically on all systems; for example, consider 64-bit systems with a userspace
process running in 32-bit mode. As a result, there is strong pressure to implement
miscellaneous control operations by just about any other means. Possible alternatives
include embedding commands into the data stream (we will discuss this
approach later in this chapter) or using virtual filesystems, either sysfs or driver-specific
filesystems. (We will look at sysfs in Chapter 14.) However, the fact remains
that ioctl is often the easiest and most straightforward choice for true device operations.
The ioctl driver method has a prototype that differs somewhat from the user-space
version:
int (*ioctl) (struct inode *inode, struct file *filp,
unsigned int cmd, unsigned long arg);
The inode and filp pointers are the values corresponding to the file
descriptor fd passed on by the application and are the same parameters passed to the open
method. The cmd argument is passed from the user unchanged, and the optional arg
argument is passed in the form of an unsigned long, regardless of whether it was
given by the user as an integer or a pointer. If the invoking program doesn't pass a
third argument, the arg value received by the driver operation is undefined. Because
type checking is disabled on the extra argument, the compiler can't warn you if an
invalid argument is passed to ioctl, and any associated bug would be difficult to spot.
As you might imagine, most ioctl implementations consist of a big switch statement
that selects the correct behavior according to the cmd argument. Different commands
have different numeric values, which are usually given symbolic names to simplify
coding. The symbolic name is assigned by a preprocessor definition. Custom drivers
usually declare such symbols in their header files; scull.h declares them for scull. User
PAGE 137
programs must, of course, include that header file as well to have access to those
symbols.
Choosing the ioctl Commands
Before writing the code for ioctl, you need to choose the numbers that correspond to
commands. The first instinct of many programmers is to choose a set of small numbers
starting with 0 or 1 and going up from there. There are, however, good reasons
for not doing things that way. The ioctl command numbers should be unique across
the system in order to prevent errors caused by issuing the right command to the
wrong device. Such a mismatch is not unlikely to happen, and a program might find
itself trying to change the baud rate of a non-serial-port input stream, such as a FIFO
or an audio device. If each ioctl number is unique, the application gets an EINVAL
error rather than succeeding in doing something unintended.
To help programmers create unique ioctl command codes, these codes have been
split up into several bitfields. The first versions of Linux used 16-bit numbers: the
top eight were the "magic" numbers associated with the device, and the bottom eight
were a sequential number, unique within the device. This happened because Linus
was "clueless" (his own word); a better division of bitfields was conceived only later.
Unfortunately, quite a few drivers still use the old convention. They have to: changing
the command codes would break no end of binary programs, and that is not
something the kernel developers are willing to do.
To choose ioctl numbers for your driver according to the Linux kernel convention,
you should first check include/asm/ioctl.h and Documentation/ioctl-number.txt. The
header defines the bitfields you will be using: type (magic number), ordinal number,
direction of transfer, and size of argument. The ioctl-number.txt file lists the magic
numbers used throughout the kernel,* so you'll be able to choose your own magic
number and avoid overlaps. The text file also lists the reasons why the convention
should be used.
The approved way to define ioctl command numbers uses four bitfields, which have
the following meanings. New symbols introduced in this list are defined in <linux/
ioctl.h>.
type
The magic number. Just choose one number (after consulting ioctl-number.txt)
and use it throughout the driver. This field is eight bits wide (_IOC_TYPEBITS).
number
The ordinal (sequential) number. It's eight bits (_IOC_NRBITS) wide.
* Maintenance of this file has been somewhat scarce as of late, however.
PAGE 138
direction
The direction of data transfer, if the particular command involves a data transfer.
The possible values are _IOC_NONE (no data transfer), _IOC_READ, _IOC_WRITE,
and _IOC_READ|_IOC_WRITE (data is transferred both ways). Data transfer is seen
from the application's point of view; _IOC_READ means reading from the device,
so the driver must write to user space. Note that the field is a bit mask, so _IOC_READ
and _IOC_WRITE can be extracted using a logical AND operation.
size
The size of user data involved. The width of this field is architecture dependent,
but is usually 13 or 14 bits. You can find its value for your specific architecture
in the macro _IOC_SIZEBITS. It's not mandatory that you use the size field--the
kernel does not check it--but it is a good idea. Proper use of this field can help
detect user-space programming errors and enable you to implement backward
compatibility if you ever need to change the size of the relevant data item. If you
need larger data structures, however, you can just ignore the size field. We'll see
how this field is used soon.
The header file <asm/ioctl.h>, which is included by <linux/ioctl.h>, defines macros
that help set up the command numbers as follows: _IO(type,nr) (for a command
that has no argument), _IOR(type,nr,datatype) (for reading data from the
driver), _IOW(type,nr,datatype) (for writing data), and _IOWR(type,nr,datatype) (for
bidirectional transfers). The type and number fields are passed as arguments, and the
size field is derived by applying sizeof to the datatype argument.
The header also defines macros that may be used in your driver to decode the numbers:
_IOC_DIR(nr), _IOC_TYPE(nr), _IOC_NR(nr), and _IOC_SIZE(nr). We won't go
into any more detail about these macros because the header file is clear, and sample
code is shown later in this section.
Here is how some ioctl commands are defined in scull. In particular, these commands
set and get the driver's configurable parameters.
/* Use 'k' as magic number */
#define SCULL_IOC_MAGIC 'k'
/* Please use a different 8-bit number in your code */
#define SCULL_IOCRESET _IO(SCULL_IOC_MAGIC, 0)
/*
* S means "Set" through a ptr,
* T means "Tell" directly with the argument value
* G means "Get": reply by setting through a pointer
* Q means "Query": response is on the return value
* X means "eXchange": switch G and S atomically
* H means "sHift": switch T and Q atomically
*/
#define SCULL_IOCSQUANTUM _IOW(SCULL_IOC_MAGIC, 1, int)
#define SCULL_IOCSQSET _IOW(SCULL_IOC_MAGIC, 2, int)
PAGE 139
#define SCULL_IOCTQUANTUM _IO(SCULL_IOC_MAGIC, 3)
#define SCULL_IOCTQSET _IO(SCULL_IOC_MAGIC, 4)
#define SCULL_IOCGQUANTUM _IOR(SCULL_IOC_MAGIC, 5, int)
#define SCULL_IOCGQSET _IOR(SCULL_IOC_MAGIC, 6, int)
#define SCULL_IOCQQUANTUM _IO(SCULL_IOC_MAGIC, 7)
#define SCULL_IOCQQSET _IO(SCULL_IOC_MAGIC, 8)
#define SCULL_IOCXQUANTUM _IOWR(SCULL_IOC_MAGIC, 9, int)
#define SCULL_IOCXQSET _IOWR(SCULL_IOC_MAGIC,10, int)
#define SCULL_IOCHQUANTUM _IO(SCULL_IOC_MAGIC, 11)
#define SCULL_IOCHQSET _IO(SCULL_IOC_MAGIC, 12)
#define SCULL_IOC_MAXNR 14
The actual source file defines a few extra commands that have not been shown here.
We chose to implement both ways of passing integer arguments: by pointer and by
explicit value (although, by an established convention, ioctl should exchange values
by pointer). Similarly, both ways are used to return an integer number: by pointer or
by setting the return value. This works as long as the return value is a positive integer;
as you know by now, on return from any system call, a positive value is preserved
(as we saw for read and write), while a negative value is considered an error
and is used to set errno in user space.*
The "exchange" and "shift" operations are not particularly useful for scull. We
implemented "exchange" to show how the driver can combine separate operations
into a single atomic one, and "shift" to pair "tell" and "query." There are times when
atomic test-and-set operations like these are needed, in particular, when applications
need to set or release locks.
The explicit ordinal number of the command has no specific meaning. It is used only
to tell the commands apart. Actually, you could even use the same ordinal number
for a read command and a write command, since the actual ioctl number is different
in the "direction" bits, but there is no reason why you would want to do so. We
chose not to use the ordinal number of the command anywhere but in the declaration,
so we didn't assign a symbolic value to it. That's why explicit numbers appear
in the definition given previously. The example shows one way to use the command
numbers, but you are free to do it differently.
With the exception of a small number of predefined commands (to be discussed
shortly), the value of the ioctl cmd argument is not currently used by the kernel, and
it's quite unlikely it will be in the future. Therefore, you could, if you were feeling
lazy, avoid the complex declarations shown earlier and explicitly declare a set of scalar
numbers. On the other hand, if you did, you wouldn't benefit from using the bitfields,
and you would encounter difficulties if you ever submitted your code for
* Actually, all libc implementations currently in use (including uClibc) consider as
error codes only values in the range -4095 to -1. Unfortunately, being able to return large
negative numbers but not small ones is not very useful.
PAGE 140
inclusion in the mainline kernel. The header <linux/kd.h> is an example of this old-fashioned
approach, using 16-bit scalar values to define the ioctl commands. That
source file relied on scalar numbers because it used the conventions obeyed at that
time, not out of laziness. Changing it now would cause gratuitous incompatibility.
The Return Value
The implementation of ioctl is usually a switch statement based on the command
number. But what should the default selection be when the command number
doesn't match a valid operation? The question is controversial. Several kernel functions
return -EINVAL ("Invalid argument"), which makes sense because the command
argument is indeed not a valid one. The POSIX standard, however, states that
if an inappropriate ioctl command has been issued, then -ENOTTY should be returned.
This error code is interpreted by the C library as "inappropriate ioctl for device,"
which is usually exactly what the programmer needs to hear. It's still pretty common,
though, to return -EINVAL in response to an invalid ioctl command.
The Predefined Commands
Although the ioctl system call is most often used to act on devices, a few commands
are recognized by the kernel. Note that these commands, when applied to your
device, are decoded before your own file operations are called. Thus, if you choose
the same number for one of your ioctl commands, you won't ever see any request for
that command, and the application gets something unexpected because of the conflict
between the ioctl numbers.
The predefined commands are divided into three groups:
- Those that can be issued on any file (regular, device, FIFO, or socket)
- Those that are issued only on regular files
- Those specific to the filesystem type
Commands in the last group are executed by the implementation of the hosting filesystem
(this is how the chattr command works). Device driver writers are interested
only in the first group of commands, whose magic number is "T." Looking at the
workings of the other groups is left to the reader as an exercise; ext2_ioctl is a most
interesting function (and easier to understand than one might expect), because it
implements the append-only flag and the immutable flag.
PAGE 141
The following ioctl commands are predefined for any file, including device-special
files:
FIOCLEX
Set the close-on-exec flag (File IOctl CLose on EXec). Setting this flag causes the
file descriptor to be closed when the calling process executes a new program.
FIONCLEX
Clear the close-on-exec flag (File IOctl Not CLos on EXec). The command
restores the common file behavior, undoing what FIOCLEX above does.
FIOASYNC
Set or reset asynchronous notification for the file (as discussed in the section
"Asynchronous Notification," later in this chapter). Note that kernel versions up
to Linux 2.2.4 incorrectly used this command to modify the O_SYNC flag. Since
both actions can be accomplished through fcntl, nobody actually uses the
FIOASYNC command, which is reported here only for completeness.
FIOQSIZE
This command returns the size of a file or directory; when applied to a device
file, however, it yields an ENOTTY error return.
FIONBIO
"File IOctl Non-Blocking I/O" (described in the section "Blocking and Nonblocking
Operations"). This call modifies the O_NONBLOCK flag in filp->f_flags.
The third argument to the system call is used to indicate whether the flag is to be
set or cleared. (We'll look at the role of the flag later in this chapter.) Note that
the usual way to change this flag is with the fcntl system call, using the F_SETFL
command.
The last item in the list introduced a new system call, fcntl, which looks like ioctl. In
fact, the fcntl call is very similar to ioctl in that it gets a command argument and an
extra (optional) argument. It is kept separate from ioctl mainly for historical reasons:
when Unix developers faced the problem of controlling I/O operations, they decided
that files and devices were different. At the time, the only devices with ioctl implementations
were ttys, which explains why -ENOTTY is the standard reply for an incorrect
ioctl command. Things have changed, but fcntl remains a separate system call.
Using the ioctl Argument
Another point we need to cover before looking at the ioctl code for the scull driver is
how to use the extra argument. If it is an integer, it's easy: it can be used directly. If it
is a pointer, however, some care must be taken.
PAGE 142
When a pointer is used to refer to user space, we must ensure that the user address is
valid. An attempt to access an unverified user-supplied pointer can lead to incorrect
behavior, a kernel oops, system corruption, or security problems. It is the driver's
responsibility to make proper checks on every user-space address it uses and to
return an error if it is invalid.
In Chapter 3, we looked at the copy_from_user and copy_to_user functions, which
can be used to safely move data to and from user space. Those functions can be used
in ioctl methods as well, but ioctl calls often involve small data items that can be
more efficiently manipulated through other means. To start, address verification
(without transferring data) is implemented by the function access_ok, which is
declared in <asm/uaccess.h>:
int access_ok(int type, const void *addr, unsigned long size);
The first argument should be either VERIFY_READ or VERIFY_WRITE, depending on
whether the action to be performed is reading the user-space memory area or writing
it. The addr argument holds a user-space address, and size is a byte count. If ioctl,
for instance, needs to read an integer value from user space, size is sizeof(int). If
you need to both read and write at the given address, use VERIFY_WRITE, since it is a
superset of VERIFY_READ.
Unlike most kernel functions, access_ok returns a boolean value: 1 for success (access
is OK) and 0 for failure (access is not OK). If it returns false, the driver should usually
return -EFAULT to the caller.
There are a couple of interesting things to note about access_ok. First, it does not do
the complete job of verifying memory access; it only checks to see that the memory
reference is in a region of memory that the process might reasonably have access to.
In particular, access_ok ensures that the address does not point to kernel-space memory.
Second, most driver code need not actually call access_ok. The memory-access
routines described later take care of that for you. Nonetheless, we demonstrate its
use so that you can see how it is done.
The scull source exploits the bitfields in the ioctl number to check the arguments
before the switch:
int err = 0, tmp;
int retval = 0;
/*
* extract the type and number bitfields, and don't decode
* wrong cmds: return ENOTTY (inappropriate ioctl) before access_ok( )
*/
if (_IOC_TYPE(cmd) != SCULL_IOC_MAGIC) return -ENOTTY;
if (_IOC_NR(cmd) > SCULL_IOC_MAXNR) return -ENOTTY;
/*
* the direction is a bitmask, and VERIFY_WRITE catches R/W
* transfers. `Type' is user-oriented, while
PAGE 143
* access_ok is kernel-oriented, so the concept of "read" and
* "write" is reversed
*/
if (_IOC_DIR(cmd) & _IOC_READ)
err = !access_ok(VERIFY_WRITE, (void __user *)arg, _IOC_SIZE(cmd));
else if (_IOC_DIR(cmd) & _IOC_WRITE)
err = !access_ok(VERIFY_READ, (void __user *)arg, _IOC_SIZE(cmd));
if (err) return -EFAULT;
After calling access_ok, the driver can safely perform the actual transfer. In addition
to the copy_from_user and copy_to_user functions, the programmer can exploit a set
of functions that are optimized for the most used data sizes (one, two, four, and eight
bytes). These functions are described in the following list and are defined in <asm/uaccess.h>:
put_user(datum, ptr)
__put_user(datum, ptr)
These macros write the datum to user space; they are relatively fast and should be
called instead of copy_to_user whenever single values are being transferred. The
macros have been written to allow the passing of any type of pointer to put_user,
as long as it is a user-space address. The size of the data transfer depends on the
type of the ptr argument and is determined at compile time using the sizeof and
typeof compiler builtins. As a result, if ptr is a char pointer, one byte is transferred,
and so on for two, four, and possibly eight bytes.
put_user checks to ensure that the process is able to write to the given memory
address. It returns 0 on success, and -EFAULT on error. __put_user performs less
checking (it does not call access_ok), but can still fail if the memory pointed to is
not writable by the user. Thus, __put_user should only be used if the memory
region has already been verified with access_ok.
As a general rule, you call __put_user to save a few cycles when you are implementing
a read method, or when you copy several items and, thus, call access_ok
just once before the first data transfer, as shown above for ioctl.
get_user(local, ptr)
__get_user(local, ptr)
These macros are used to retrieve a single datum from user space. They behave
like put_user and __put_user, but transfer data in the opposite direction. The
value retrieved is stored in the local variable local; the return value indicates
whether the operation succeeded. Again, __get_user should only be used if the
address has already been verified with access_ok.
If an attempt is made to use one of the listed functions to transfer a value that does
not fit one of the specific sizes, the result is usually a strange message from the compiler,
such as "conversion to non-scalar type requested." In such cases, copy_to_user
or copy_from_user must be used.
PAGE 144
Capabilities and Restricted Operations
Access to a device is controlled by the permissions on the device file(s), and the driver
is not normally involved in permissions checking. There are situations, however,
where any user is granted read/write permission on the device, but some control operations
should still be denied. For example, not all users of a tape drive should be able
to set its default block size, and a user who has been granted read/write access to a
disk device should probably still be denied the ability to format it. In cases like these,
the driver must perform additional checks to be sure that the user is capable of performing
the requested operation.
Unix systems have traditionally restricted privileged operations to the superuser
account. This meant that privilege was an all-or-nothing thing--the superuser can do
absolutely anything, but all other users are highly restricted. The Linux kernel provides
a more flexible system called capabilities. A capability-based system leaves the
all-or-nothing mode behind and breaks down privileged operations into separate
subgroups. In this way, a particular user (or program) can be empowered to perform
a specific privileged operation without giving away the ability to perform other, unrelated
operations. The kernel uses capabilities exclusively for permissions management
and exports two system calls capget and capset, to allow them to be managed
from user space.
The full set of capabilities can be found in <linux/capability.h>. These are the only
capabilities known to the system; it is not possible for driver authors or system administrators
to define new ones without modifying the kernel source. A subset of those
capabilities that might be of interest to device driver writers includes the following:
CAP_DAC_OVERRIDE
The ability to override access restrictions (data access control, or DAC) on files
and directories.
CAP_NET_ADMIN
The ability to perform network administration tasks, including those that affect
network interfaces.
CAP_SYS_MODULE
The ability to load or remove kernel modules.
CAP_SYS_RAWIO
The ability to perform "raw" I/O operations. Examples include accessing device
ports or communicating directly with USB devices.
CAP_SYS_ADMIN
A catch-all capability that provides access to many system administration operations.
CAP_SYS_TTY_CONFIG
The ability to perform tty configuration tasks.
PAGE 145
Before performing a privileged operation, a device driver should check that the calling
process has the appropriate capability; failure to do so could result user processes
performing unauthorized operations with bad results on system stability or
security. Capability checks are performed with the capable function (defined in
<linux/sched.h>):
int capable(int capability);
In the scull sample driver, any user is allowed to query the quantum and quantum set
sizes. Only privileged users, however, may change those values, since inappropriate
values could badly affect system performance. When needed, the scull implementation
of ioctl checks a user's privilege level as follows:
if (! capable (CAP_SYS_ADMIN))
return -EPERM;
In the absence of a more specific capability for this task, CAP_SYS_ADMIN was chosen
for this test.
The Implementation of the ioctl Commands
The scull implementation of ioctl only transfers the configurable parameters of the
device and turns out to be as easy as the following:
switch(cmd) {
case SCULL_IOCRESET:
scull_quantum = SCULL_QUANTUM;
scull_qset = SCULL_QSET;
break;
case SCULL_IOCSQUANTUM: /* Set: arg points to the value */
if (! capable (CAP_SYS_ADMIN))
return -EPERM;
retval = __get_user(scull_quantum, (int __user *)arg);
break;
case SCULL_IOCTQUANTUM: /* Tell: arg is the value */
if (! capable (CAP_SYS_ADMIN))
return -EPERM;
scull_quantum = arg;
break;
case SCULL_IOCGQUANTUM: /* Get: arg is pointer to result */
retval = __put_user(scull_quantum, (int __user *)arg);
break;
case SCULL_IOCQQUANTUM: /* Query: return it (it's positive) */
return scull_quantum;
case SCULL_IOCXQUANTUM: /* eXchange: use arg as pointer */
if (! capable (CAP_SYS_ADMIN))
PAGE 146
return -EPERM;
tmp = scull_quantum;
retval = __get_user(scull_quantum, (int __user *)arg);
if (retval = = 0)
retval = __put_user(tmp, (int __user *)arg);
break;
case SCULL_IOCHQUANTUM: /* sHift: like Tell + Query */
if (! capable (CAP_SYS_ADMIN))
return -EPERM;
tmp = scull_quantum;
scull_quantum = arg;
return tmp;
default: /* redundant, as cmd was checked against MAXNR */
return -ENOTTY;
}
return retval;
scull also includes six entries that act on scull_qset. These entries are identical to the
ones for scull_quantum and are not worth showing in print.
The six ways to pass and receive arguments look like the following from the caller's
point of view (i.e., from user space):
int quantum;
ioctl(fd,SCULL_IOCSQUANTUM, &quantum); /* Set by pointer */
ioctl(fd,SCULL_IOCTQUANTUM, quantum); /* Set by value */
ioctl(fd,SCULL_IOCGQUANTUM, &quantum); /* Get by pointer */
quantum = ioctl(fd,SCULL_IOCQQUANTUM); /* Get by return value */
ioctl(fd,SCULL_IOCXQUANTUM, &quantum); /* Exchange by pointer */
quantum = ioctl(fd,SCULL_IOCHQUANTUM, quantum); /* Exchange by value */
Of course, a normal driver would not implement such a mix of calling modes. We
have done so here only to demonstrate the different ways in which things could be
done. Normally, however, data exchanges would be consistently performed, either
through pointers or by value, and mixing of the two techniques would be avoided.
Device Control Without ioctl
Sometimes controlling the device is better accomplished by writing control
sequences to the device itself. For example, this technique is used in the console
driver, where so-called escape sequences are used to move the cursor, change the
default color, or perform other configuration tasks. The benefit of implementing
device control this way is that the user can control the device just by writing data,
without needing to use (or sometimes write) programs built just for configuring the
device. When devices can be controlled in this manner, the program issuing commands
often need not even be running on the same system as the device it is controlling.
PAGE 147
For example, the setterm program acts on the console (or another terminal) configuration
by printing escape sequences. The controlling program can live on a different
computer from the controlled device, because a simple redirection of the data stream
does the configuration job. This is what happens every time you run a remote tty session:
escape sequences are printed remotely but affect the local tty; the technique is
not restricted to ttys, though.
The drawback of controlling by printing is that it adds policy constraints to the
device; for example, it is viable only if you are sure that the control sequence can't
appear in the data being written to the device during normal operation. This is only
partly true for ttys. Although a text display is meant to display only ASCII characters,
sometimes control characters can slip through in the data being written and
can, therefore, affect the console setup. This can happen, for example, when you cat
a binary file to the screen; the resulting mess can contain anything, and you often
end up with the wrong font on your console.
Controlling by write is definitely the way to go for those devices that don't transfer
data but just respond to commands, such as robotic devices.
For instance, a driver written for fun by one of your authors moves a camera on two
axes. In this driver, the "device" is simply a pair of old stepper motors, which can't
really be read from or written to. The concept of "sending a data stream" to a stepper
motor makes little or no sense. In this case, the driver interprets what is being
written as ASCII commands and converts the requests to sequences of impulses that
manipulate the stepper motors. The idea is similar, somewhat, to the AT commands
you send to the modem in order to set up communication, the main difference being
that the serial port used to communicate with the modem must transfer real data as
well. The advantage of direct device control is that you can use cat to move the camera
without writing and compiling special code to issue the ioctl calls.
When writing command-oriented drivers, there's no reason to implement the ioctl
method. An additional command in the interpreter is easier to implement and use.
Sometimes, though, you might choose to act the other way around: instead of turning
the write method into an interpreter and avoiding ioctl, you might choose to
avoid write altogether and use ioctl commands exclusively, while accompanying the
driver with a specific command-line tool to send those commands to the driver. This
approach moves the complexity from kernel space to user space, where it may be
easier to deal with, and helps keep the driver small while denying use of simple cat or
echo commands.
Blocking I/O
Back in Chapter 3, we looked at how to implement the read and write driver methods.
At that point, however, we skipped over one important issue: how does a driver
respond if it cannot immediately satisfy the request? A call to read may come when
PAGE 148
no data is available, but more is expected in the future. Or a process could attempt to
write, but your device is not ready to accept the data, because your output buffer is
full. The calling process usually does not care about such issues; the programmer
simply expects to call read or write and have the call return after the necessary work
has been done. So, in such cases, your driver should (by default) block the process,
putting it to sleep until the request can proceed.
This section shows how to put a process to sleep and wake it up again later on. As
usual, however, we have to explain a few concepts first.
Introduction to Sleeping
What does it mean for a process to "sleep"? When a process is put to sleep, it is
marked as being in a special state and removed from the scheduler's run queue. Until
something comes along to change that state, the process will not be scheduled on
any CPU and, therefore, will not run. A sleeping process has been shunted off to the
side of the system, waiting for some future event to happen.
Causing a process to sleep is an easy thing for a Linux device driver to do. There are,
however, a couple of rules that you must keep in mind to be able to code sleeps in a
safe manner.
The first of these rules is: never sleep when you are running in an atomic context.
We got an introduction to atomic operation in Chapter 5; an atomic context is simply
a state where multiple steps must be performed without any sort of concurrent
access. What that means, with regard to sleeping, is that your driver cannot sleep
while holding a spinlock, seqlock, or RCU lock. You also cannot sleep if you have
disabled interrupts. It is legal to sleep while holding a semaphore, but you should
look very carefully at any code that does so. If code sleeps while holding a semaphore,
any other thread waiting for that semaphore also sleeps. So any sleeps that
happen while holding semaphores should be short, and you should convince yourself
that, by holding the semaphore, you are not blocking the process that will eventually
wake you up.
Another thing to remember with sleeping is that, when you wake up, you never
know how long your process may have been out of the CPU or what may have
changed in the mean time. You also do not usually know if another process may
have been sleeping for the same event; that process may wake before you and grab
whatever resource you were waiting for. The end result is that you can make no
assumptions about the state of the system after you wake up, and you must check to
ensure that the condition you were waiting for is, indeed, true.
One other relevant point, of course, is that your process cannot sleep unless it is
assured that somebody else, somewhere, will wake it up. The code doing the awakening
must also be able to find your process to be able to do its job. Making sure that
a wakeup happens is a matter of thinking through your code and knowing, for each
PAGE 149
sleep, exactly what series of events will bring that sleep to an end. Making it possible
for your sleeping process to be found is, instead, accomplished through a data structure
called a wait queue. A wait queue is just what it sounds like: a list of processes,
all waiting for a specific event.
In Linux, a wait queue is managed by means of a "wait queue head," a structure of
type wait_queue_head_t, which is defined in <linux/wait.h>. A wait queue head can
be defined and initialized statically with:
DECLARE_WAIT_QUEUE_HEAD(name);
or dynamically as follows:
wait_queue_head_t my_queue;
init_waitqueue_head(&my_queue);
We will return to the structure of wait queues shortly, but we know enough now to
take a first look at sleeping and waking up.
Simple Sleeping
When a process sleeps, it does so in expectation that some condition will become
true in the future. As we noted before, any process that sleeps must check to be sure
that the condition it was waiting for is really true when it wakes up again. The simplest
way of sleeping in the Linux kernel is a macro called wait_event (with a few
variants); it combines handling the details of sleeping with a check on the condition
a process is waiting for. The forms of wait_event are:
wait_event(queue, condition)
wait_event_interruptible(queue, condition)
wait_event_timeout(queue, condition, timeout)
wait_event_interruptible_timeout(queue, condition, timeout)
In all of the above forms, queue is the wait queue head to use. Notice that it is passed
"by value." The condition is an arbitrary boolean expression that is evaluated by the
macro before and after sleeping; until condition evaluates to a true value, the process
continues to sleep. Note that condition may be evaluated an arbitrary number of
times, so it should not have any side effects.
If you use wait_event, your process is put into an uninterruptible sleep which, as we
have mentioned before, is usually not what you want. The preferred alternative is
wait_event_interruptible, which can be interrupted by signals. This version returns an
integer value that you should check; a nonzero value means your sleep was interrupted
by some sort of signal, and your driver should probably return -ERESTARTSYS.
The final versions (wait_event_timeout and wait_event_interruptible_timeout) wait for
a limited time; after that time period (expressed in jiffies, which we will discuss in
Chapter 7) expires, the macros return with a value of 0 regardless of how condition
evaluates.
PAGE 150
The other half of the picture, of course, is waking up. Some other thread of execution
(a different process, or an interrupt handler, perhaps) has to perform the
wakeup for you, since your process is, of course, asleep. The basic function that
wakes up sleeping processes is called wake_up. It comes in several forms (but we
look at only two of them now):
void wake_up(wait_queue_head_t *queue);
void wake_up_interruptible(wait_queue_head_t *queue);
wake_up wakes up all processes waiting on the given queue (though the situation is a
little more complicated than that, as we will see later). The other form (wake_up_
interruptible) restricts itself to processes performing an interruptible sleep. In general,
the two are indistinguishable (if you are using interruptible sleeps); in practice,
the convention is to use wake_up if you are using wait_event and wake_up_interruptible
if you use wait_event_interruptible.
We now know enough to look at a simple example of sleeping and waking up. In the
sample source, you can find a module called sleepy. It implements a device with simple
behavior: any process that attempts to read from the device is put to sleep.
Whenever a process writes to the device, all sleeping processes are awakened. This
behavior is implemented with the following read and write methods:
static DECLARE_WAIT_QUEUE_HEAD(wq);
static int flag = 0;
ssize_t sleepy_read (struct file *filp, char __user *buf, size_t count, loff_t *pos)
{
printk(KERN_DEBUG "process %i (%s) going to sleep\n",
current->pid, current->comm);
wait_event_interruptible(wq, flag != 0);
flag = 0;
printk(KERN_DEBUG "awoken %i (%s)\n", current->pid, current->comm);
return 0; /* EOF */
}
ssize_t sleepy_write (struct file *filp, const char __user *buf, size_t count,
loff_t *pos)
{
printk(KERN_DEBUG "process %i (%s) awakening the readers...\n",
current->pid, current->comm);
flag = 1;
wake_up_interruptible(&wq);
return count; /* succeed, to avoid retrial */
}
Note the use of the flag variable in this example. Since wait_event_interruptible
checks for a condition that must become true, we use flag to create that condition.
It is interesting to consider what happens if two processes are waiting when sleepy_write
is called. Since sleepy_read resets flag to 0 once it wakes up, you might think that the
second process to wake up would immediately go back to sleep. On a single-processor
PAGE 151
system, that is almost always what happens. But it is important to understand why you
cannot count on that behavior. The wake_up_interruptible call will cause both sleeping
processes to wake up. It is entirely possible that they will both note that flag is nonzero
before either has the opportunity to reset it. For this trivial module, this race condition is
unimportant. In a real driver, this kind of race can create rare crashes that are difficult to
diagnose. If correct operation required that exactly one process see the nonzero value, it
would have to be tested in an atomic manner. We will see how a real driver handles
such situations shortly. But first we have to cover one other topic.
Blocking and Nonblocking Operations
One last point we need to touch on before we look at the implementation of full-featured
read and write methods is deciding when to put a process to sleep. There are
times when implementing proper Unix semantics requires that an operation not
block, even if it cannot be completely carried out.
There are also times when the calling process informs you that it does not want to
block, whether or not its I/O can make any progress at all. Explicitly nonblocking I/O
is indicated by the O_NONBLOCK flag in filp->f_flags. The flag is defined in <linux/
fcntl.h>, which is automatically included by <linux/fs.h>. The flag gets its name from
"open-nonblock," because it can be specified at open time (and originally could be
specified only there). If you browse the source code, you find some references to an
O_NDELAY flag; this is an alternate name for O_NONBLOCK, accepted for compatibility
with System V code. The flag is cleared by default, because the normal behavior of a
process waiting for data is just to sleep. In the case of a blocking operation, which is
the default, the following behavior should be implemented in order to adhere to the
standard semantics:
- If a process calls read but no data is (yet) available, the process must block. The
process is awakened as soon as some data arrives, and that data is returned to
the caller, even if there is less than the amount requested in the count argument
to the method.
- If a process calls write and there is no space in the buffer, the process must
block, and it must be on a different wait queue from the one used for reading.
When some data has been written to the hardware device, and space becomes
free in the output buffer, the process is awakened and the write call succeeds,
although the data may be only partially written if there isn't room in the buffer
for the count bytes that were requested.
Both these statements assume that there are both input and output buffers; in practice,
almost every device driver has them. The input buffer is required to avoid losing
data that arrives when nobody is reading. In contrast, data can't be lost on write,
because if the system call doesn't accept data bytes, they remain in the user-space
buffer. Even so, the output buffer is almost always useful for squeezing more performance
out of the hardware.
PAGE 152
The performance gain of implementing an output buffer in the driver results from
the reduced number of context switches and user-level/kernel-level transitions.
Without an output buffer (assuming a slow device), only one or a few characters are
accepted by each system call, and while one process sleeps in write, another process
runs (that's one context switch). When the first process is awakened, it resumes
(another context switch), write returns (kernel/user transition), and the process reiterates
the system call to write more data (user/kernel transition); the call blocks and
the loop continues. The addition of an output buffer allows the driver to accept
larger chunks of data with each write call, with a corresponding increase in performance.
If that buffer is big enough, the write call succeeds on the first attempt--the
buffered data will be pushed out to the device later--without control needing to go
back to user space for a second or third write call. The choice of a suitable size for
the output buffer is clearly device-specific.
We don't use an input buffer in scull, because data is already available when read is
issued. Similarly, no output buffer is used, because data is simply copied to the memory
area associated with the device. Essentially, the device is a buffer, so the implementation
of additional buffers would be superfluous. We'll see the use of buffers in
Chapter 10.
The behavior of read and write is different if O_NONBLOCK is specified. In this case, the
calls simply return -EAGAIN ("try it again") if a process calls read when no data is
available or if it calls write when there's no space in the buffer.
As you might expect, nonblocking operations return immediately, allowing the
application to poll for data. Applications must be careful when using the stdio functions
while dealing with nonblocking files, because they can easily mistake a nonblocking
return for EOF. They always have to check errno.
Naturally, O_NONBLOCK is meaningful in the open method also. This happens when the
call can actually block for a long time; for example, when opening (for read access) a
FIFO that has no writers (yet), or accessing a disk file with a pending lock. Usually,
opening a device either succeeds or fails, without the need to wait for external
events. Sometimes, however, opening the device requires a long initialization, and
you may choose to support O_NONBLOCK in your open method by returning immediately
with -EAGAIN if the flag is set, after starting the device initialization process. The
driver may also implement a blocking open to support access policies in a way similar
to file locks. We'll see one such implementation in the section "Blocking open as
an Alternative to EBUSY" later in this chapter.
Some drivers may also implement special semantics for O_NONBLOCK; for example, an
open of a tape device usually blocks until a tape has been inserted. If the tape drive is
opened with O_NONBLOCK, the open succeeds immediately regardless of whether the
media is present or not.
Only the read, write, and open file operations are affected by the nonblocking flag.
PAGE 153
A Blocking I/O Example
Finally, we get to an example of a real driver method that implements blocking I/O.
This example is taken from the scullpipe driver; it is a special form of scull that implements
a pipe-like device.
Within a driver, a process blocked in a read call is awakened when data arrives; usually
the hardware issues an interrupt to signal such an event, and the driver awakens
waiting processes as part of handling the interrupt. The scullpipe driver works differently,
so that it can be run without requiring any particular hardware or an interrupt
handler. We chose to use another process to generate the data and wake the reading
process; similarly, reading processes are used to wake writer processes that are waiting
for buffer space to become available.
The device driver uses a device structure that contains two wait queues and a buffer.
The size of the buffer is configurable in the usual ways (at compile time, load time, or
runtime).
struct scull_pipe {
wait_queue_head_t inq, outq; /* read and write queues */
char *buffer, *end; /* begin of buf, end of buf */
int buffersize; /* used in pointer arithmetic */
char *rp, *wp; /* where to read, where to write */
int nreaders, nwriters; /* number of openings for r/w */
struct fasync_struct *async_queue; /* asynchronous readers */
struct semaphore sem; /* mutual exclusion semaphore */
struct cdev cdev; /* Char device structure */
};
The read implementation manages both blocking and nonblocking input and looks
like this:
static ssize_t scull_p_read (struct file *filp, char __user *buf, size_t count,
loff_t *f_pos)
{
struct scull_pipe *dev = filp->private_data;
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
while (dev->rp = = dev->wp) { /* nothing to read */
up(&dev->sem); /* release the lock */
if (filp->f_flags & O_NONBLOCK)
return -EAGAIN;
PDEBUG("\"%s\" reading: going to sleep\n", current->comm);
if (wait_event_interruptible(dev->inq, (dev->rp != dev->wp)))
return -ERESTARTSYS; /* signal: tell the fs layer to handle it */
/* otherwise loop, but first reacquire the lock */
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
}
/* ok, data is there, return something */
PAGE 154
if (dev->wp > dev->rp)
count = min(count, (size_t)(dev->wp - dev->rp));
else /* the write pointer has wrapped, return data up to dev->end */
count = min(count, (size_t)(dev->end - dev->rp));
if (copy_to_user(buf, dev->rp, count)) {
up (&dev->sem);
return -EFAULT;
}
dev->rp += count;
if (dev->rp = = dev->end)
dev->rp = dev->buffer; /* wrapped */
up (&dev->sem);
/* finally, awake any writers and return */
wake_up_interruptible(&dev->outq);
PDEBUG("\"%s\" did read %li bytes\n",current->comm, (long)count);
return count;
}
As you can see, we left some PDEBUG statements in the code. When you compile the
driver, you can enable messaging to make it easier to follow the interaction of different
processes.
Let us look carefully at how scull_p_read handles waiting for data. The while loop
tests the buffer with the device semaphore held. If there is data there, we know we
can return it to the user immediately without sleeping, so the entire body of the loop
is skipped. If, instead, the buffer is empty, we must sleep. Before we can do that,
however, we must drop the device semaphore; if we were to sleep holding it, no
writer would ever have the opportunity to wake us up. Once the semaphore has been
dropped, we make a quick check to see if the user has requested non-blocking I/O,
and return if so. Otherwise, it is time to call wait_event_interruptible.
Once we get past that call, something has woken us up, but we do not know what.
One possibility is that the process received a signal. The if statement that contains
the wait_event_interruptible call checks for this case. This statement ensures the
proper and expected reaction to signals, which could have been responsible for waking
up the process (since we were in an interruptible sleep). If a signal has arrived
and it has not been blocked by the process, the proper behavior is to let upper layers
of the kernel handle the event. To this end, the driver returns -ERESTARTSYS to the
caller; this value is used internally by the virtual filesystem (VFS) layer, which either
restarts the system call or returns -EINTR to user space. We use the same type of
check to deal with signal handling for every read and write implementation.
However, even in the absence of a signal, we do not yet know for sure that there is
data there for the taking. Somebody else could have been waiting for data as well,
and they might win the race and get the data first. So we must acquire the device
semaphore again; only then can we test the read buffer again (in the while loop) and
truly know that we can return the data in the buffer to the user. The end result of all
PAGE 155
this code is that, when we exit from the while loop, we know that the semaphore is
held and the buffer contains data that we can use.
Just for completeness, let us note that scull_p_read can sleep in another spot after we
take the device semaphore: the call to copy_to_user. If scull sleeps while copying data
between kernel and user space, it sleeps with the device semaphore held. Holding the
semaphore in this case is justified since it does not deadlock the system (we know
that the kernel will perform the copy to user space and wakes us up without trying to
lock the same semaphore in the process), and since it is important that the device
memory array not change while the driver sleeps.
Advanced Sleeping
Many drivers are able to meet their sleeping requirements with the functions we have
covered so far. There are situations, however, that call for a deeper understanding of
how the Linux wait queue mechanism works. Complex locking or performance
requirements can force a driver to use lower-level functions to effect a sleep. In this
section, we look at the lower level to get an understanding of what is really going on
when a process sleeps.
How a process sleeps
If you look inside <linux/wait.h>, you see that the data structure behind
the wait_queue_head_t type is quite simple; it consists of a spinlock and a linked list. What goes
on to that list is a wait queue entry, which is declared with the type wait_queue_t. This
structure contains information about the sleeping process and exactly how it would
like to be woken up.
The first step in putting a process to sleep is usually the allocation and initialization
of a wait_queue_t structure, followed by its addition to the proper wait queue. When
everything is in place, whoever is charged with doing the wakeup will be able to find
the right processes.
The next step is to set the state of the process to mark it as being asleep. There are several
task states defined in <linux/sched.h>. TASK_RUNNING means that the process is able
to run, although it is not necessarily executing in the processor at any specific moment.
There are two states that indicate that a process is asleep: TASK_INTERRUPTIBLE and
TASK_UNINTERRUPTIBLE; they correspond, of course, to the two types of sleep. The other
states are not normally of concern to driver writers.
In the 2.6 kernel, it is not normally necessary for driver code to manipulate the process
state directly. However, should you need to do so, the call to use is:
void set_current_state(int new_state);
In older code, you often see something like this instead:
current->state = TASK_INTERRUPTIBLE;
PAGE 156
But changing current directly in that manner is discouraged; such code breaks easily
when data structures change. The above code does show, however, that changing the
current state of a process does not, by itself, put it to sleep. By changing the current
state, you have changed the way the scheduler treats a process, but you have not yet
yielded the processor.
Giving up the processor is the final step, but there is one thing to do first: you must
check the condition you are sleeping for first. Failure to do this check invites a race
condition; what happens if the condition came true while you were engaged in the
above process, and some other thread has just tried to wake you up? You could miss
the wakeup altogether and sleep longer than you had intended. Consequently, down
inside code that sleeps, you typically see something such as:
if (!condition)
schedule( );
By checking our condition after setting the process state, we are covered against all
possible sequences of events. If the condition we are waiting for had come about
before setting the process state, we notice in this check and not actually sleep. If the
wakeup happens thereafter, the process is made runnable whether or not we have
actually gone to sleep yet.
The call to schedule is, of course, the way to invoke the scheduler and yield the CPU.
Whenever you call this function, you are telling the kernel to consider which process
should be running and to switch control to that process if necessary. So you never
know how long it will be before schedule returns to your code.
After the if test and possible call to (and return from) schedule, there is some
cleanup to be done. Since the code no longer intends to sleep, it must ensure that the
task state is reset to TASK_RUNNING. If the code just returned from schedule, this step is
unnecessary; that function does not return until the process is in a runnable state.
But if the call to schedule was skipped because it was no longer necessary to sleep,
the process state will be incorrect. It is also necessary to remove the process from the
wait queue, or it may be awakened more than once.
Manual sleeps
In previous versions of the Linux kernel, nontrivial sleeps required the programmer
to handle all of the above steps manually. It was a tedious process involving a fair
amount of error-prone boilerplate code. Programmers can still code a manual sleep
in that manner if they want to; <linux/sched.h> contains all the requisite definitions,
and the kernel source abounds with examples. There is an easier way, however.
The first step is the creation and initialization of a wait queue entry. That is usually
done with this macro:
DEFINE_WAIT(my_wait);
PAGE 157
in which name is the name of the wait queue entry variable. You can also do things in
two steps:
wait_queue_t my_wait;
init_wait(&my_wait);
But it is usually easier to put a DEFINE_WAIT line at the top of the loop that implements
your sleep.
The next step is to add your wait queue entry to the queue, and set the process state.
Both of those tasks are handled by this function:
void prepare_to_wait(wait_queue_head_t *queue,
wait_queue_t *wait,
int state);
Here, queue and wait are the wait queue head and the process entry, respectively.
state is the new state for the process; it should be either TASK_INTERRUPTIBLE (for
interruptible sleeps, which is usually what you want) or TASK_UNINTERRUPTIBLE (for
uninterruptible sleeps).
After calling prepare_to_wait, the process can call schedule--after it has checked to
be sure it still needs to wait. Once schedule returns, it is cleanup time. That task, too,
is handled by a special function:
void finish_wait(wait_queue_head_t *queue, wait_queue_t *wait);
Thereafter, your code can test its state and see if it needs to wait again.
We are far past due for an example. Previously we looked at the read method for
scullpipe, which uses wait_event. The write method in the same driver does its waiting
with prepare_to_wait and finish_wait, instead. Normally you would not mix
methods within a single driver in this way, but we did so in order to be able to show
both ways of handling sleeps.
First, for completeness, let's look at the write method itself:
/* How much space is free? */
static int spacefree(struct scull_pipe *dev)
{
if (dev->rp = = dev->wp)
return dev->buffersize - 1;
return ((dev->rp + dev->buffersize - dev->wp) % dev->buffersize) - 1;
}
static ssize_t scull_p_write(struct file *filp, const char __user *buf, size_t count,
loff_t *f_pos)
{
struct scull_pipe *dev = filp->private_data;
int result;
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
PAGE 158
/* Make sure there's space to write */
result = scull_getwritespace(dev, filp);
if (result)
return result; /* scull_getwritespace called up(&dev->sem) */
/* ok, space is there, accept something */
count = min(count, (size_t)spacefree(dev));
if (dev->wp >= dev->rp)
count = min(count, (size_t)(dev->end - dev->wp)); /* to end-of-buf */
else /* the write pointer has wrapped, fill up to rp-1 */
count = min(count, (size_t)(dev->rp - dev->wp - 1));
PDEBUG("Going to accept %li bytes to %p from %p\n", (long)count, dev->wp, buf);
if (copy_from_user(dev->wp, buf, count)) {
up (&dev->sem);
return -EFAULT;
}
dev->wp += count;
if (dev->wp = = dev->end)
dev->wp = dev->buffer; /* wrapped */
up(&dev->sem);
/* finally, awake any reader */
wake_up_interruptible(&dev->inq); /* blocked in read( ) and select( ) */
/* and signal asynchronous readers, explained late in chapter 5 */
if (dev->async_queue)
kill_fasync(&dev->async_queue, SIGIO, POLL_IN);
PDEBUG("\"%s\" did write %li bytes\n",current->comm, (long)count);
return count;
}
This code looks similar to the read method, except that we have pushed the code
that sleeps into a separate function called scull_getwritespace. Its job is to ensure that
there is space in the buffer for new data, sleeping if need be until that space comes
available. Once the space is there, scull_p_write can simply copy the user's data
there, adjust the pointers, and wake up any processes that may have been waiting to
read data.
The code that handles the actual sleep is:
/* Wait for space for writing; caller must hold device semaphore. On
* error the semaphore will be released before returning. */
static int scull_getwritespace(struct scull_pipe *dev, struct file *filp)
{
while (spacefree(dev) = = 0) { /* full */
DEFINE_WAIT(wait);
up(&dev->sem);
if (filp->f_flags & O_NONBLOCK)
return -EAGAIN;
PAGE 159
PDEBUG("\"%s\" writing: going to sleep\n",current->comm);
prepare_to_wait(&dev->outq, &wait, TASK_INTERRUPTIBLE);
if (spacefree(dev) = = 0)
schedule( );
finish_wait(&dev->outq, &wait);
if (signal_pending(current))
return -ERESTARTSYS; /* signal: tell the fs layer to handle it */
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
}
return 0;
}
Note once again the containing while loop. If space is available without sleeping, this
function simply returns. Otherwise, it must drop the device semaphore and wait.
The code uses DEFINE_WAIT to set up a wait queue entry and prepare_to_wait to
get ready for the actual sleep. Then comes the obligatory check on the buffer; we
must handle the case in which space becomes available in the buffer after we have
entered the while loop (and dropped the semaphore) but before we put ourselves
onto the wait queue. Without that check, if the reader processes were able to completely
empty the buffer in that time, we could miss the only wakeup we would ever
get and sleep forever. Having satisfied ourselves that we must sleep, we can call
schedule.
It is worth looking again at this case: what happens if the wakeup happens between
the test in the if statement and the call to schedule? In that case, all is well. The
wakeup resets the process state to TASK_RUNNING and schedule returns--although not
necessarily right away. As long as the test happens after the process has put itself on
the wait queue and changed its state, things will work.
To finish up, we call finish_wait. The call to signal_pending tells us whether we were
awakened by a signal; if so, we need to return to the user and let them try again later.
Otherwise, we reacquire the semaphore, and test again for free space as usual.
Exclusive waits
We have seen that when a process calls wake_up on a wait queue, all processes waiting
on that queue are made runnable. In many cases, that is the correct behavior. In
others, however, it is possible to know ahead of time that only one of the processes
being awakened will succeed in obtaining the desired resource, and the rest will simply
have to sleep again. Each one of those processes, however, has to obtain the processor,
contend for the resource (and any governing locks), and explicitly go back to
sleep. If the number of processes in the wait queue is large, this "thundering herd"
behavior can seriously degrade the performance of the system.
PAGE 160
In response to real-world thundering herd problems, the kernel developers added an
"exclusive wait" option to the kernel. An exclusive wait acts very much like a normal
sleep, with two important differences:
- When a wait queue entry has the WQ_FLAG_EXCLUSIVE flag set, it is added to the end
of the wait queue. Entries without that flag are, instead, added to the beginning.
- When wake_up is called on a wait queue, it stops after waking the first process
that has the WQ_FLAG_EXCLUSIVE flag set.
The end result is that processes performing exclusive waits are awakened one at a
time, in an orderly manner, and do not create thundering herds. The kernel still
wakes up all nonexclusive waiters every time, however.
Employing exclusive waits within a driver is worth considering if two conditions are
met: you expect significant contention for a resource, and waking a single process is
sufficient to completely consume the resource when it becomes available. Exclusive
waits work well for the Apache web server, for example; when a new connection
comes in, exactly one of the (often many) Apache processes on the system should
wake up to deal with it. We did not use exclusive waits in the scullpipe driver, however;
it is rare to see readers contending for data (or writers for buffer space), and we
cannot know that one reader, once awakened, will consume all of the available data.
Putting a process into an interruptible wait is a simple matter of calling prepare_to_
wait_exclusive:
void prepare_to_wait_exclusive(wait_queue_head_t *queue,
wait_queue_t *wait,
int state);
This call, when used in place of prepare_to_wait, sets the "exclusive" flag in the wait
queue entry and adds the process to the end of the wait queue. Note that there is no
way to perform exclusive waits with wait_event and its variants.
The details of waking up
The view we have presented of the wakeup process is simpler than what really happens
inside the kernel. The actual behavior that results when a process is awakened
is controlled by a function in the wait queue entry. The default wakeup function* sets
the process into a runnable state and, possibly, performs a context switch to that
process if it has a higher priority. Device drivers should never need to supply a different
wake function; should yours prove to be the exception, see <linux/wait.h> for
information on how to do it.
* It has the imaginative name default_wake_function.
PAGE 161
We have not yet seen all the variations of wake_up. Most driver writers never need
the others, but, for completeness, here is the full set:
wake_up(wait_queue_head_t *queue);
wake_up_interruptible(wait_queue_head_t *queue);
wake_up awakens every process on the queue that is not in an exclusive wait,
and exactly one exclusive waiter, if it exists. wake_up_interruptible does the
same, with the exception that it skips over processes in an uninterruptible sleep.
These functions can, before returning, cause one or more of the processes awakened
to be scheduled (although this does not happen if they are called from an
atomic context).
wake_up_nr(wait_queue_head_t *queue, int nr);
wake_up_interruptible_nr(wait_queue_head_t *queue, int nr);
These functions perform similarly to wake_up, except they can awaken up to nr
exclusive waiters, instead of just one. Note that passing 0 is interpreted as asking
for all of the exclusive waiters to be awakened, rather than none of them.
wake_up_all(wait_queue_head_t *queue);
wake_up_interruptible_all(wait_queue_head_t *queue);
This form of wake_up awakens all processes whether they are performing an
exclusive wait or not (though the interruptible form still skips processes doing
uninterruptible waits).
wake_up_interruptible_sync(wait_queue_head_t *queue);
Normally, a process that is awakened may preempt the current process and be
scheduled into the processor before wake_up returns. In other words, a call to
wake_up may not be atomic. If the process calling wake_up is running in an
atomic context (it holds a spinlock, for example, or is an interrupt handler), this
rescheduling does not happen. Normally, that protection is adequate. If, however,
you need to explicitly ask to not be scheduled out of the processor at this
time, you can use the "sync" variant of wake_up_interruptible. This function is
most often used when the caller is about to reschedule anyway, and it is more
efficient to simply finish what little work remains first.
If all of the above is not entirely clear on a first reading, don't worry. Very few drivers
ever need to call anything except wake_up_interruptible.
Ancient history: sleep_on
If you spend any time digging through the kernel source, you will likely encounter
two functions that we have neglected to discuss so far:
void sleep_on(wait_queue_head_t *queue);
void interruptible_sleep_on(wait_queue_head_t *queue);
As you might expect, these functions unconditionally put the current process to
sleep on the given queue. These functions are strongly deprecated, however, and you
PAGE 162
should never use them. The problem is obvious if you think about it: sleep_on offers
no way to protect against race conditions. There is always a window between when
your code decides it must sleep and when sleep_on actually effects that sleep. A
wakeup that arrives during that window is missed. For this reason, code that calls
sleep_on is never entirely safe.
Current plans call for sleep_on and its variants (there are a couple of time-out forms
we haven't shown) to be removed from the kernel in the not-too-distant future.
Testing the Scullpipe Driver
We have seen how the scullpipe driver implements blocking I/O. If you wish to try it
out, the source to this driver can be found with the rest of the book examples. Blocking
I/O in action can be seen by opening two windows. The first can run a command such
as cat /dev/scullpipe. If you then, in another window, copy a file to /dev/scullpipe, you
should see that file's contents appear in the first window.
Testing nonblocking activity is trickier, because the conventional programs available
to a shell don't perform nonblocking operations. The misc-progs source directory
contains the following simple program, called nbtest, for testing nonblocking operations.
All it does is copy its input to its output, using nonblocking I/O and delaying
between retries. The delay time is passed on the command line and is one second by
default.
int main(int argc, char **argv)
{
int delay = 1, n, m = 0;
if (argc > 1)
delay=atoi(argv[1]);
fcntl(0, F_SETFL, fcntl(0,F_GETFL) | O_NONBLOCK); /* stdin */
fcntl(1, F_SETFL, fcntl(1,F_GETFL) | O_NONBLOCK); /* stdout */
while (1) {
n = read(0, buffer, 4096);
if (n >= 0)
m = write(1, buffer, n);
if ((n < 0 || m < 0) && (errno != EAGAIN))
break;
sleep(delay);
}
perror(n < 0 ? "stdin" : "stdout");
exit(1);
}
If you run this program under a process tracing utility such as strace, you can see the
success or failure of each operation, depending on whether data is available when the
operation is tried.
PAGE 163
poll and select
Applications that use nonblocking I/O often use the poll, select, and epoll system
calls as well. poll, select, and epoll have essentially the same functionality: each allow
a process to determine whether it can read from or write to one or more open files
without blocking. These calls can also block a process until any of a given set of file
descriptors becomes available for reading or writing. Therefore, they are often used
in applications that must use multiple input or output streams without getting stuck
on any one of them. The same functionality is offered by multiple functions, because
two were implemented in Unix almost at the same time by two different groups:
select was introduced in BSD Unix, whereas poll was the System V solution. The epoll
call* was added in 2.5.45 as a way of making the polling function scale to thousands
of file descriptors.
Support for any of these calls requires support from the device driver. This support
(for all three calls) is provided through the driver's poll method. This method has the
following prototype:
unsigned int (*poll) (struct file *filp, poll_table *wait);
The driver method is called whenever the user-space program performs a poll, select,
or epoll system call involving a file descriptor associated with the driver. The device
method is in charge of these two steps:
- Call poll_wait on one or more wait queues that could indicate a change in the
poll status. If no file descriptors are currently available for I/O, the kernel causes
the process to wait on the wait queues for all file descriptors passed to the system call.
- Return a bit mask describing the operations (if any) that could be immediately
performed without blocking.
Both of these operations are usually straightforward and tend to look very similar
from one driver to the next. They rely, however, on information that only the driver
can provide and, therefore, must be implemented individually by each driver.
The poll_table structure, the second argument to the poll method, is used within the
kernel to implement the poll, select, and epoll calls; it is declared in <linux/poll.h>,
which must be included by the driver source. Driver writers do not need to know
anything about its internals and must use it as an opaque object; it is passed to the
driver method so that the driver can load it with every wait queue that could wake
up the process and change the status of the poll operation. The driver adds a wait
queue to the poll_table structure by calling the function poll_wait:
void poll_wait (struct file *, wait_queue_head_t *, poll_table *);
* Actually, epoll is a set of three calls that together can be used to achieve the polling functionality. For our
purposes, though, we can think of it as a single call.
PAGE 164
The second task performed by the poll method is returning the bit mask describing
which operations could be completed immediately; this is also straightforward. For
example, if the device has data available, a read would complete without sleeping;
the poll method should indicate this state of affairs. Several flags (defined via <linux/
poll.h>) are used to indicate the possible operations:
POLLIN
This bit must be set if the device can be read without blocking.
POLLRDNORM
This bit must be set if "normal" data is available for reading. A readable device
returns (POLLIN | POLLRDNORM).
POLLRDBAND
This bit indicates that out-of-band data is available for reading from the device.
It is currently used only in one place in the Linux kernel (the DECnet code) and
is not generally applicable to device drivers.
POLLPRI
High-priority data (out-of-band) can be read without blocking. This bit causes
select to report that an exception condition occurred on the file, because select
reports out-of-band data as an exception condition.
POLLHUP
When a process reading this device sees end-of-file, the driver must set POLLHUP
(hang-up). A process calling select is told that the device is readable, as dictated
by the select functionality.
POLLERR
An error condition has occurred on the device. When poll is invoked, the device
is reported as both readable and writable, since both read and write return an
error code without blocking.
POLLOUT
This bit is set in the return value if the device can be written to without blocking.
POLLWRNORM
This bit has the same meaning as POLLOUT, and sometimes it actually is the same number. A writable device returns (POLLOUT | POLLWRNORM).
POLLWRBAND
Like POLLRDBAND, this bit means that data with nonzero priority can be written to
the device. Only the datagram implementation of poll uses this bit, since a datagram
can transmit out-of-band data.
It's worth repeating that POLLRDBAND and POLLWRBAND are meaningful only with file
descriptors associated with sockets: device drivers won't normally use these flags.
PAGE 165
The description of poll takes up a lot of space for something that is relatively simple
to use in practice. Consider the scullpipe implementation of the poll method:
static unsigned int scull_p_poll(struct file *filp, poll_table *wait)
{
struct scull_pipe *dev = filp->private_data;
unsigned int mask = 0;
/*
* The buffer is circular; it is considered full
* if "wp" is right behind "rp" and empty if the
* two are equal.
*/
down(&dev->sem);
poll_wait(filp, &dev->inq, wait);
poll_wait(filp, &dev->outq, wait);
if (dev->rp != dev->wp)
mask |= POLLIN | POLLRDNORM; /* readable */
if (spacefree(dev))
mask |= POLLOUT | POLLWRNORM; /* writable */
up(&dev->sem);
return mask;
}
This code simply adds the two scullpipe wait queues to the poll_table, then sets the
appropriate mask bits depending on whether data can be read or written.
The poll code as shown is missing end-of-file support, because scullpipe does not
support an end-of-file condition. For most real devices, the poll method should
return POLLHUP if no more data is (or will become) available. If the caller used the
select system call, the file is reported as readable. Regardless of whether poll or select
is used, the application knows that it can call read without waiting forever, and the
read method returns, 0 to signal end-of-file.
With real FIFOs, for example, the reader sees an end-of-file when all the writers close
the file, whereas in scullpipe the reader never sees end-of-file. The behavior is different
because a FIFO is intended to be a communication channel between two processes,
while scullpipe is a trash can where everyone can put data as long as there's at
least one reader. Moreover, it makes no sense to reimplement what is already available
in the kernel, so we chose to implement a different behavior in our example.
Implementing end-of-file in the same way as FIFOs do would mean checking
dev->nwriters, both in read and in poll, and reporting end-of-file (as just described) if no
process has the device opened for writing. Unfortunately, though, with this implementation,
if a reader opened the scullpipe device before the writer, it would see endof-file
without having a chance to wait for data. The best way to fix this problem
would be to implement blocking within open like real FIFOs do; this task is left as an
exercise for the reader.
PAGE 166
Interaction with read and write
The purpose of the poll and select calls is to determine in advance if an I/O operation
will block. In that respect, they complement read and write. More important, poll
and select are useful, because they let the application wait simultaneously for several
data streams, although we are not exploiting this feature in the scull examples.
A correct implementation of the three calls is essential to make applications work
correctly: although the following rules have more or less already been stated, we
summarize them here.
Reading data from the device
- If there is data in the input buffer, the read call should return immediately, with
no noticeable delay, even if less data is available than the application requested,
and the driver is sure the remaining data will arrive soon. You can always return
less data than you're asked for if this is convenient for any reason (we did it in
scull), provided you return at least one byte. In this case, poll should return
POLLIN|POLLRDNORM.
- If there is no data in the input buffer, by default read must block until at least
one byte is there. If O_NONBLOCK is set, on the other hand, read returns immediately
with a return value of -EAGAIN (although some old versions of System V
return 0 in this case). In these cases, poll must report that the device is unreadable
until at least one byte arrives. As soon as there is some data in the buffer, we
fall back to the previous case.
- If we are at end-of-file, read should return immediately with a return value of 0,
independent of O_NONBLOCK. poll should report POLLHUP in this case.
Writing to the device
- If there is space in the output buffer, write should return without delay. It can
accept less data than the call requested, but it must accept at least one byte. In
this case, poll reports that the device is writable by returning POLLOUT|POLLWRNORM.
- If the output buffer is full, by default write blocks until some space is freed. If
O_NONBLOCK is set, write returns immediately with a return value of -EAGAIN (older
System V Unices returned 0). In these cases, poll should report that the file is not
writable. If, on the other hand, the device is not able to accept any more data,
write returns -ENOSPC ("No space left on device"), independently of the setting of
O_NONBLOCK.
- Never make a write call wait for data transmission before returning, even if
O_NONBLOCK is clear. This is because many applications use select to find out
whether a write will block. If the device is reported as writable, the call must not
block. If the program using the device wants to ensure that the data it enqueues
PAGE 167
in the output buffer is actually transmitted, the driver must provide an fsync
method. For instance, a removable device should have an fsync entry point.
Although this is a good set of general rules, one should also recognize that each
device is unique and that sometimes the rules must be bent slightly. For example,
record-oriented devices (such as tape drives) cannot execute partial writes.
Flushing pending output
We've seen how the write method by itself doesn't account for all data output needs.
The fsync function, invoked by the system call of the same name, fills the gap. This
method's prototype is
int (*fsync) (struct file *file, struct dentry *dentry, int datasync);
If some application ever needs to be assured that data has been sent to the device, the
fsync method must be implemented regardless of whether O_NONBLOCK is set. A call to
fsync should return only when the device has been completely flushed (i.e., the output
buffer is empty), even if that takes some time. The datasync argument is used to
distinguish between the fsync and fdatasync system calls; as such, it is only of interest
to filesystem code and can be ignored by drivers.
The fsync method has no unusual features. The call isn't time critical, so every device
driver can implement it to the author's taste. Most of the time, char drivers just have
a NULL pointer in their fops. Block devices, on the other hand, always implement the
method with the general-purpose block_fsync, which, in turn, flushes all the blocks
of the device, waiting for I/O to complete.
The Underlying Data Structure
The actual implementation of the poll and select system calls is reasonably simple, for
those who are interested in how it works; epoll is a bit more complex but is built on the
same mechanism. Whenever a user application calls poll, select, or epoll_ctl,* the kernel
invokes the poll method of all files referenced by the system call, passing the same
poll_table to each of them. The poll_table structure is just a wrapper around a function
that builds the actual data structure. That structure, for poll and select, is a linked
list of memory pages containing poll_table_entry structures. Each poll_table_entry
holds the struct file and wait_queue_head_t pointers passed to poll_wait, along with
an associated wait queue entry. The call to poll_wait sometimes also adds the process
to the given wait queue. The whole structure must be maintained by the kernel so that
the process can be removed from all of those queues before poll or select returns.
If none of the drivers being polled indicates that I/O can occur without blocking, the
poll call simply sleeps until one of the (perhaps many) wait queues it is on wakes it up.
* This is the function that sets up the internal data structure for future calls to epoll_wait.
PAGE 168
What's interesting in the implementation of poll is that the driver's poll method may
be called with a NULL pointer as a poll_table argument. This situation can come
about for a couple of reasons. If the application calling poll has provided a timeout
value of 0 (indicating that no wait should be done), there is no reason to accumulate
wait queues, and the system simply does not do it. The poll_table pointer is also set
to NULL immediately after any driver being polled indicates that I/O is possible. Since
the kernel knows at that point that no wait will occur, it does not build up a list of
wait queues.
When the poll call completes, the poll_table structure is deallocated, and all wait
queue entries previously added to the poll table (if any) are removed from the table
and their wait queues.
We tried to show the data structures involved in polling in Figure 6-1; the figure is
a simplified representation of the real data structures, because it ignores the multipage
nature of a poll table and disregards the file pointer that is part of each
poll_table_entry. The reader interested in the actual implementation is urged to
look in <linux/poll.h> and fs/select.c.
Figure 6-1. The data structures behind poll
PAGE 169
At this point, it is possible to understand the motivation behind the new epoll system
call. In a typical case, a call to poll or select involves only a handful of file
descriptors, so the cost of setting up the data structure is small. There are applications
out there, however, that work with thousands of file descriptors. At that point,
setting up and tearing down this data structure between every I/O operation
becomes prohibitively expensive. The epoll system call family allows this sort of
application to set up the internal kernel data structure exactly once and to use it
many times.
Asynchronous Notification
Although the combination of blocking and nonblocking operations and the select
method are sufficient for querying the device most of the time, some situations aren't
efficiently managed by the techniques we've seen so far.
Let's imagine a process that executes a long computational loop at low priority but
needs to process incoming data as soon as possible. If this process is responding to
new observations available from some sort of data acquisition peripheral, it would
like to know immediately when new data is available. This application could be written
to call poll regularly to check for data, but, for many situations, there is a better
way. By enabling asynchronous notification, this application can receive a signal
whenever data becomes available and need not concern itself with polling.
User programs have to execute two steps to enable asynchronous notification from
an input file. First, they specify a process as the "owner" of the file. When a process
invokes the F_SETOWN command using the fcntl system call, the process ID of the
owner process is saved in filp->f_owner for later use. This step is necessary for the
kernel to know just whom to notify. In order to actually enable asynchronous notification,
the user programs must set the FASYNC flag in the device by means of the
F_SETFL fcntl command.
After these two calls have been executed, the input file can request delivery of a SIGIO
signal whenever new data arrives. The signal is sent to the process (or process group,
if the value is negative) stored in filp->f_owner.
For example, the following lines of code in a user program enable asynchronous
notification to the current process for the stdin input file:
signal(SIGIO, &input_handler); /* dummy sample; sigaction( ) is better */
fcntl(STDIN_FILENO, F_SETOWN, getpid( ));
oflags = fcntl(STDIN_FILENO, F_GETFL);
fcntl(STDIN_FILENO, F_SETFL, oflags | FASYNC);
The program named asynctest in the sources is a simple program that reads stdin as
shown. It can be used to test the asynchronous capabilities of scullpipe. The program
is similar to cat but doesn't terminate on end-of-file; it responds only to input, not to
the absence of input.
PAGE 170
Note, however, that not all the devices support asynchronous notification, and you
can choose not to offer it. Applications usually assume that the asynchronous capability
is available only for sockets and ttys.
There is one remaining problem with input notification. When a process receives a
SIGIO, it doesn't know which input file has new input to offer. If more than one file is
enabled to asynchronously notify the process of pending input, the application must
still resort to poll or select to find out what happened.
The Driver's Point of View
A more relevant topic for us is how the device driver can implement asynchronous
signaling. The following list details the sequence of operations from the kernel's
point of view:
- When F_SETOWN is invoked, nothing happens, except that a value is assigned to
filp->f_owner.
- When F_SETFL is executed to turn on FASYNC, the driver's fasync method is called.
This method is called whenever the value of FASYNC is changed in filp->f_flags
to notify the driver of the change, so it can respond properly. The flag is cleared
by default when the file is opened. We'll look at the standard implementation of
the driver method later in this section.
- When data arrives, all the processes registered for asynchronous notification
must be sent a SIGIO signal.
While implementing the first step is trivial--there's nothing to do on the driver's
part--the other steps involve maintaining a dynamic data structure to keep track of
the different asynchronous readers; there might be several. This dynamic data structure,
however, doesn't depend on the particular device involved, and the kernel
offers a suitable general-purpose implementation so that you don't have to rewrite
the same code in every driver.
The general implementation offered by Linux is based on one data structure and two
functions (which are called in the second and third steps described earlier). The
header that declares related material is <linux/fs.h> (nothing new here), and the data
structure is called struct fasync_struct. As with wait queues, we need to insert a
pointer to the structure in the device-specific data structure.
The two functions that the driver calls correspond to the following prototypes:
int fasync_helper(int fd, struct file *filp,
int mode, struct fasync_struct **fa);
void kill_fasync(struct fasync_struct **fa, int sig, int band);
fasync_helper is invoked to add or remove entries from the list of interested processes
when the FASYNC flag changes for an open file. All of its arguments except the last are
provided to the fasync method and can be passed through directly. kill_fasync is used
PAGE 171
to signal the interested processes when data arrives. Its arguments are the signal to
send (usually SIGIO) and the band, which is almost always POLL_IN* (but that may be
used to send "urgent" or out-of-band data in the networking code).
Here's how scullpipe implements the fasync method:
static int scull_p_fasync(int fd, struct file *filp, int mode)
{
struct scull_pipe *dev = filp->private_data;
return fasync_helper(fd, filp, mode, &dev->async_queue);
}
It's clear that all the work is performed by fasync_helper. It wouldn't be possible,
however, to implement the functionality without a method in the driver, because the
helper function needs to access the correct pointer to struct fasync_struct * (here
&dev->async_queue), and only the driver can provide the information.
When data arrives, then, the following statement must be executed to signal asynchronous
readers. Since new data for the scullpipe reader is generated by a process
issuing a write, the statement appears in the write method of scullpipe.
if (dev->async_queue)
kill_fasync(&dev->async_queue, SIGIO, POLL_IN);
Note that some devices also implement asynchronous notification to indicate when
the device can be written; in this case, of course, kill_fasync must be called with a
mode of POLL_OUT.
It might appear that we're done, but there's still one thing missing. We must invoke
our fasync method when the file is closed to remove the file from the list of active
asynchronous readers. Although this call is required only if filp->f_flags has FASYNC
set, calling the function anyway doesn't hurt and is the usual implementation. The
following lines, for example, are part of the release method for scullpipe:
/* remove this filp from the asynchronously notified filp's */
scull_p_fasync(-1, filp, 0);
The data structure underlying asynchronous notification is almost identical to the
structure struct wait_queue, because both situations involve waiting on an event.
The difference is that struct file is used in place of struct task_struct.
The struct file in the queue is then used to retrieve f_owner, in order to signal the process.
Seeking a Device
One of the last things we need to cover in this chapter is the llseek method, which is
useful (for some devices) and easy to implement.
* POLL_IN is a symbol used in the asynchronous notification code; it is equivalent to POLLIN|POLLRDNORM.
PAGE 172
The llseek Implementation
The llseek method implements the lseek and llseek system calls. We have already
stated that if the llseek method is missing from the device's operations, the default
implementation in the kernel performs seeks by modifying filp->f_pos, the current
reading/writing position within the file. Please note that for the lseek system call to
work correctly, the read and write methods must cooperate by using and updating
the offset item they receive as an argument.
You may need to provide your own llseek method if the seek operation corresponds
to a physical operation on the device. A simple example can be seen in the scull
driver:
loff_t scull_llseek(struct file *filp, loff_t off, int whence)
{
struct scull_dev *dev = filp->private_data;
loff_t newpos;
switch(whence) {
case 0: /* SEEK_SET */
newpos = off;
break;
case 1: /* SEEK_CUR */
newpos = filp->f_pos + off;
break;
case 2: /* SEEK_END */
newpos = dev->size + off;
break;
default: /* can't happen */
return -EINVAL;
}
if (newpos < 0) return -EINVAL;
filp->f_pos = newpos;
return newpos;
}
The only device-specific operation here is retrieving the file length from the device. In
scull the read and write methods cooperate as needed, as shown in Chapter 3.
Although the implementation just shown makes sense for scull, which handles a well-defined
data area, most devices offer a data flow rather than a data area (just think
about the serial ports or the keyboard), and seeking those devices does not make
sense. If this is the case for your device, you can't just refrain from declaring the llseek
operation, because the default method allows seeking. Instead, you should inform the
kernel that your device does not support llseek by calling nonseekable_open in your
open method:
int nonseekable_open(struct inode *inode; struct file *filp);
PAGE 173
This call marks the given filp as being nonseekable; the kernel never allows an lseek
call on such a file to succeed. By marking the file in this way, you can also be assured
that no attempts will be made to seek the file by way of the pread and pwrite system
calls.
For completeness, you should also set the llseek method in your file_operations
structure to the special helper function no_llseek, which is defined in <linux/fs.h>.
Access Control on a Device File
Offering access control is sometimes vital for the reliability of a device node. Not
only should unauthorized users not be permitted to use the device (a restriction is
enforced by the filesystem permission bits), but sometimes only one authorized user
should be allowed to open the device at a time.
The problem is similar to that of using ttys. In that case, the login process changes
the ownership of the device node whenever a user logs into the system, in order to
prevent other users from interfering with or sniffing the tty data flow. However, it's
impractical to use a privileged program to change the ownership of a device every
time it is opened just to grant unique access to it.
None of the code shown up to now implements any access control beyond the filesystem
permission bits. If the open system call forwards the request to the driver,
open succeeds. We now introduce a few techniques for implementing some additional
checks.
Every device shown in this section has the same behavior as the bare scull device
(that is, it implements a persistent memory area) but differs from scull in access control,
which is implemented in the open and release operations.
Single-Open Devices
The brute-force way to provide access control is to permit a device to be opened by
only one process at a time (single openness). This technique is best avoided because it
inhibits user ingenuity. A user might want to run different processes on the same
device, one reading status information while the other is writing data. In some cases,
users can get a lot done by running a few simple programs through a shell script, as
long as they can access the device concurrently. In other words, implementing a singleopen
behavior amounts to creating policy, which may get in the way of what your
users want to do.
Allowing only a single process to open a device has undesirable properties, but it is
also the easiest access control to implement for a device driver, so it's shown here.
The source code is extracted from a device called scullsingle.
PAGE 174
The scullsingle device maintains an atomic_t variable called scull_s_available; that
variable is initialized to a value of one, indicating that the device is indeed available.
The open call decrements and tests scull_s_available and refuses access if somebody
else already has the device open:
static atomic_t scull_s_available = ATOMIC_INIT(1);
static int scull_s_open(struct inode *inode, struct file *filp)
{
struct scull_dev *dev = &scull_s_device; /* device information */
if (! atomic_dec_and_test (&scull_s_available)) {
atomic_inc(&scull_s_available);
return -EBUSY; /* already open */
}
/* then, everything else is copied from the bare scull device */
if ( (filp->f_flags & O_ACCMODE) = = O_WRONLY)
scull_trim(dev);
filp->private_data = dev;
return 0; /* success */
}
The release call, on the other hand, marks the device as no longer busy:
static int scull_s_release(struct inode *inode, struct file *filp)
{
atomic_inc(&scull_s_available); /* release the device */
return 0;
}
Normally, we recommend that you put the open flag scull_s_available within the
device structure (Scull_Dev here) because, conceptually, it belongs to the device. The
scull driver, however, uses stand alone variables to hold the flag so it can use the same
device structure and methods as the bare scull device and minimize code duplication.
Restricting Access to a Single User at a Time
The next step beyond a single-open device is to let a single user open a device in multiple
processes but allow only one user to have the device open at a time. This solution
makes it easy to test the device, since the user can read and write from several
processes at once, but assumes that the user takes some responsibility for maintaining
the integrity of the data during multiple accesses. This is accomplished by adding
checks in the open method; such checks are performed after the normal
permission checking and can only make access more restrictive than that specified by
the owner and group permission bits. This is the same access policy as that used for
ttys, but it doesn't resort to an external privileged program.
Those access policies are a little trickier to implement than single-open policies. In
this case, two items are needed: an open count and the uid of the "owner" of the
PAGE 175
device. Once again, the best place for such items is within the device structure; our
example uses global variables instead, for the reason explained earlier for scullsingle.
The name of the device is sculluid.
The open call grants access on first open but remembers the owner of the device.
This means that a user can open the device multiple times, thus allowing cooperating
processes to work concurrently on the device. At the same time, no other user
can open it, thus avoiding external interference. Since this version of the function is
almost identical to the preceding one, only the relevant part is reproduced here:
spin_lock(&scull_u_lock);
if (scull_u_count &&
(scull_u_owner != current->uid) && /* allow user */
(scull_u_owner != current->euid) && /* allow whoever did su */
!capable(CAP_DAC_OVERRIDE)) { /* still allow root */
spin_unlock(&scull_u_lock);
return -EBUSY; /* -EPERM would confuse the user */
}
if (scull_u_count = = 0)
scull_u_owner = current->uid; /* grab it */
scull_u_count++;
spin_unlock(&scull_u_lock);
Note that the sculluid code has two variables (scull_u_owner and scull_u_count)
that control access to the device and that could be accessed concurrently by multiple
processes. To make these variables safe, we control access to them with a spinlock
(scull_u_lock). Without that locking, two (or more) processes could test
scull_u_count at the same time, and both could conclude that they were entitled to
take ownership of the device. A spinlock is indicated here, because the lock is held
for a very short time, and the driver does nothing that could sleep while holding the
lock.
We chose to return -EBUSY and not -EPERM, even though the code is performing a permission
check, in order to point a user who is denied access in the right direction.
The reaction to "Permission denied" is usually to check the mode and owner of the
/dev file, while "Device busy" correctly suggests that the user should look for a process
already using the device.
This code also checks to see if the process attempting the open has the ability to
override file access permissions; if so, the open is allowed even if the opening process
is not the owner of the device. The CAP_DAC_OVERRIDE capability fits the task well
in this case.
The release method looks like the following:
static int scull_u_release(struct inode *inode, struct file *filp)
{
spin_lock(&scull_u_lock);
scull_u_count--; /* nothing else */
PAGE 176
spin_unlock(&scull_u_lock);
return 0;
}
Once again, we must obtain the lock prior to modifying the count to ensure that we
do not race with another process.
Blocking open as an Alternative to EBUSY
When the device isn't accessible, returning an error is usually the most sensible
approach, but there are situations in which the user would prefer to wait for the
device.
For example, if a data communication channel is used both to transmit reports on a
regular, scheduled basis (using crontab) and for casual usage according to people's
needs, it's much better for the scheduled operation to be slightly delayed rather than
fail just because the channel is currently busy.
This is one of the choices that the programmer must make when designing a device
driver, and the right answer depends on the particular problem being solved.
The alternative to EBUSY, as you may have guessed, is to implement blocking open.
The scullwuid device is a version of sculluid that waits for the device on open instead
of returning -EBUSY. It differs from sculluid only in the following part of the open
operation:
spin_lock(&scull_w_lock);
while (! scull_w_available( )) {
spin_unlock(&scull_w_lock);
if (filp->f_flags & O_NONBLOCK) return -EAGAIN;
if (wait_event_interruptible (scull_w_wait, scull_w_available( )))
return -ERESTARTSYS; /* tell the fs layer to handle it */
spin_lock(&scull_w_lock);
}
if (scull_w_count = = 0)
scull_w_owner = current->uid; /* grab it */
scull_w_count++;
spin_unlock(&scull_w_lock);
The implementation is based once again on a wait queue. If the device is not currently
available, the process attempting to open it is placed on the wait queue until
the owning process closes the device.
The release method, then, is in charge of awakening any pending process:
static int scull_w_release(struct inode *inode, struct file *filp)
{
int temp;
spin_lock(&scull_w_lock);
scull_w_count--;
temp = scull_w_count;
spin_unlock(&scull_w_lock);
PAGE 177
if (temp = = 0)
wake_up_interruptible_sync(&scull_w_wait); /* awake other uid's */
return 0;
}
Here is an example of where calling wake_up_interruptible_sync makes sense. When
we do the wakeup, we are just about to return to user space, which is a natural
scheduling point for the system. Rather than potentially reschedule when we do the
wakeup, it is better to just call the "sync" version and finish our job.
The problem with a blocking-open implementation is that it is really unpleasant for the
interactive user, who has to keep guessing what is going wrong. The interactive user
usually invokes standard commands, such as cp and tar, and can't just add O_NONBLOCK
to the open call. Someone who's making a backup using the tape drive in the next
room would prefer to get a plain "device or resource busy" message instead of being
left to guess why the hard drive is so silent today, while tar should be scanning it.
This kind of problem (a need for different, incompatible policies for the same device)
is often best solved by implementing one device node for each access policy. An
example of this practice can be found in the Linux tape driver, which provides multiple
device files for the same device. Different device files will, for example, cause the
drive to record with or without compression, or to automatically rewind the tape
when the device is closed.
Cloning the Device on open
Another technique to manage access control is to create different private copies of
the device, depending on the process opening it.
Clearly, this is possible only if the device is not bound to a hardware object; scull is
an example of such a "software" device. The internals of /dev/tty use a similar technique
in order to give its process a different "view" of what the /dev entry point represents.
When copies of the device are created by the software driver, we call them
virtual devices--just as virtual consoles use a single physical tty device.
Although this kind of access control is rarely needed, the implementation can be
enlightening in showing how easily kernel code can change the application's perspective
of the surrounding world (i.e., the computer).
The /dev/scullpriv device node implements virtual devices within the scull package.
The scullpriv implementation uses the device number of the process's controlling tty
as a key to access the virtual device. Nonetheless, you can easily modify the sources to
use any integer value for the key; each choice leads to a different policy. For example,
using the uid leads to a different virtual device for each user, while using a pid key creates
a new device for each process accessing it.
The decision to use the controlling terminal is meant to enable easy testing of the
device using I/O redirection: the device is shared by all commands run on the same
PAGE 178
virtual terminal and is kept separate from the one seen by commands run on another
terminal.
The open method looks like the following code. It must look for the right virtual
device and possibly create one. The final part of the function is not shown because it
is copied from the bare scull, which we've already seen.
/* The clone-specific data structure includes a key field */
struct scull_listitem {
struct scull_dev device;
dev_t key;
struct list_head list;
};
/* The list of devices, and a lock to protect it */
static LIST_HEAD(scull_c_list);
static spinlock_t scull_c_lock = SPIN_LOCK_UNLOCKED;
/* Look for a device or create one if missing */
static struct scull_dev *scull_c_lookfor_device(dev_t key)
{
struct scull_listitem *lptr;
list_for_each_entry(lptr, &scull_c_list, list) {
if (lptr->key = = key)
return &(lptr->device);
}
/* not found */
lptr = kmalloc(sizeof(struct scull_listitem), GFP_KERNEL);
if (!lptr)
return NULL;
/* initialize the device */
memset(lptr, 0, sizeof(struct scull_listitem));
lptr->key = key;
scull_trim(&(lptr->device)); /* initialize it */
init_MUTEX(&(lptr->device.sem));
/* place it in the list */
list_add(&lptr->list, &scull_c_list);
return &(lptr->device);
}
static int scull_c_open(struct inode *inode, struct file *filp)
{
struct scull_dev *dev;
PAGE 179
dev_t key;
if (!current->signal->tty) {
PDEBUG("Process \"%s\" has no ctl tty\n", current->comm);
return -EINVAL;
}
key = tty_devnum(current->signal->tty);
/* look for a scullc device in the list */
spin_lock(&scull_c_lock);
dev = scull_c_lookfor_device(key);
spin_unlock(&scull_c_lock);
if (!dev)
return -ENOMEM;
/* then, everything else is copied from the bare scull device */
The release method does nothing special. It would normally release the device on last
close, but we chose not to maintain an open count in order to simplify the testing of
the driver. If the device were released on last close, you wouldn't be able to read the
same data after writing to the device, unless a background process were to keep it
open. The sample driver takes the easier approach of keeping the data, so that at the
next open, you'll find it there. The devices are released when scull_cleanup is called.
This code uses the generic Linux linked list mechanism in preference to reimplementing
the same capability from scratch. Linux lists are discussed in Chapter 11.
Here's the release implementation for /dev/scullpriv, which closes the discussion of
device methods.
static int scull_c_release(struct inode *inode, struct file *filp)
{
/*
* Nothing to do, because the device is persistent.
* A `real' cloned device should be freed on last close
*/
return 0;
}
Quick Reference
This chapter introduced the following symbols and header files:
#include <linux/ioctl.h>
Declares all the macros used to define ioctl commands. It is currently included
by <linux/fs.h>.
PAGE 180
_IOC_NRBITS
_IOC_TYPEBITS
_IOC_SIZEBITS
_IOC_DIRBITS
The number of bits available for the different bitfields of ioctl commands. There
are also four macros that specify the MASKs and four that specify the SHIFTs, but
they're mainly for internal use. _IOC_SIZEBITS is an important value to check,
because it changes across architectures.
_IOC_NONE
_IOC_READ
_IOC_WRITE
The possible values for the "direction" bitfield. "Read" and "write" are different
bits and can be ORed to specify read/write. The values are 0-based.
_IOC(dir,type,nr,size)
_IO(type,nr)
_IOR(type,nr,size)
_IOW(type,nr,size)
_IOWR(type,nr,size)
Macros used to create an ioctl command.
_IOC_DIR(nr)
_IOC_TYPE(nr)
_IOC_NR(nr)
_IOC_SIZE(nr)
Macros used to decode a command. In particular, _IOC_TYPE(nr) is an OR combination
of _IOC_READ and _IOC_WRITE.
#include <asm/uaccess.h>
int access_ok(int type, const void *addr, unsigned long size);
Checks that a pointer to user space is actually usable. access_ok returns a nonzero
value if the access should be allowed.
VERIFY_READ
VERIFY_WRITE
The possible values for the type argument in access_ok. VERIFY_WRITE is a superset
of VERIFY_READ.
#include <asm/uaccess.h>
int put_user(datum,ptr);
int get_user(local,ptr);
int __put_user(datum,ptr);
int __get_user(local,ptr);
Macros used to store or retrieve a datum to or from user space. The number of
bytes being transferred depends on sizeof(*ptr). The regular versions call
PAGE 181
access_ok first, while the qualified versions (__put_user and __get_user) assume
that access_ok has already been called.
#include <linux/capability.h>
Defines the various CAP_ symbols describing the capabilities a user-space process
may have.
int capable(int capability);
Returns nonzero if the process has the given capability.
#include <linux/wait.h>
typedef struct { /* ... */ } wait_queue_head_t;
void init_waitqueue_head(wait_queue_head_t *queue);
DECLARE_WAIT_QUEUE_HEAD(queue);
The defined type for Linux wait queues. A wait_queue_head_t must be explicitly
initialized with either init_waitqueue_head at runtime or DECLARE_WAIT_
QUEUE_HEAD at compile time.
void wait_event(wait_queue_head_t q, int condition);
int wait_event_interruptible(wait_queue_head_t q, int condition);
int wait_event_timeout(wait_queue_head_t q, int condition, int time);
int wait_event_interruptible_timeout(wait_queue_head_t q, int condition, int time);
Cause the process to sleep on the given queue until the given condition evaluates
to a true value.
void wake_up(struct wait_queue **q);
void wake_up_interruptible(struct wait_queue **q);
void wake_up_nr(struct wait_queue **q, int nr);
void wake_up_interruptible_nr(struct wait_queue **q, int nr);
void wake_up_all(struct wait_queue **q);
void wake_up_interruptible_all(struct wait_queue **q);
void wake_up_interruptible_sync(struct wait_queue **q);
Wake processes that are sleeping on the queue q. The _interruptible form wakes
only interruptible processes. Normally, only one exclusive waiter is awakened,
but that behavior can be changed with the _nr or _all forms. The _sync version
does not reschedule the CPU before returning.
#include <linux/sched.h>
set_current_state(int state);
Sets the execution state of the current process. TASK_RUNNING means it is ready to
run, while the sleep states are TASK_INTERRUPTIBLE and TASK_UNINTERRUPTIBLE.
void schedule(void);
Selects a runnable process from the run queue. The chosen process can be
current or a different one.
PAGE 182
typedef struct { /* ... */ } wait_queue_t;
init_waitqueue_entry(wait_queue_t *entry, struct task_struct *task);
The wait_queue_t type is used to place a process onto a wait queue.
void prepare_to_wait(wait_queue_head_t *queue, wait_queue_t *wait, int state);
void prepare_to_wait_exclusive(wait_queue_head_t *queue, wait_queue_t *wait, int state);
void finish_wait(wait_queue_head_t *queue, wait_queue_t *wait);
Helper functions that can be used to code a manual sleep.
void sleep_on(wiat_queue_head_t *queue);
void interruptible_sleep_on(wiat_queue_head_t *queue);
Obsolete and deprecated functions that unconditionally put the current process
to sleep.
#include <linux/poll.h>
void poll_wait(struct file *filp, wait_queue_head_t *q, poll_table *p)
Places the current process into a wait queue without scheduling immediately. It
is designed to be used by the poll method of device drivers.
int fasync_helper(struct inode *inode, struct file *filp, int mode, struct
fasync_struct **fa);
A "helper" for implementing the fasync device method. The mode argument is the
same value that is passed to the method, while fa points to a device-specific
fasync_struct *.
void kill_fasync(struct fasync_struct *fa, int sig, int band);
If the driver supports asynchronous notification, this function can be used to
send a signal to processes registered in fa.
int nonseekable_open(struct inode *inode, struct file *filp);
loff_t no_llseek(struct file *file, loff_t offset, int whence);
nonseekable_open should be called in the open method of any device that does
not support seeking. Such devices should also use no_llseek as their llseek
method.
PAGE 183
CHAPTER 7
Time, Delays, and Deferred Work
At this point, we know the basics of how to write a full-featured char module. Real World
drivers, however, need to do more than implement the operations that control
a device; they have to deal with issues such as timing, memory management, hardware
access, and more. Fortunately, the kernel exports a number of facilities to ease
the task of the driver writer. In the next few chapters, we'll describe some of the kernel
resources you can use. This chapter leads the way by describing how timing
issues are addressed. Dealing with time involves the following tasks, in order of
increasing complexity:
- Measuring time lapses and comparing times
- Knowing the current time
- Delaying operation for a specified amount of time
- Scheduling asynchronous functions to happen at a later time
Measuring Time Lapses
The kernel keeps track of the flow of time by means of timer interrupts. Interrupts
are covered in detail in Chapter 10.
Timer interrupts are generated by the system's timing hardware at regular intervals;
this interval is programmed at boot time by the kernel according to the value of HZ,
which is an architecture-dependent value defined in <linux/param.h> or a subplatform
file included by it. Default values in the distributed kernel source range from 50
to 1200 ticks per second on real hardware, down to 24 for software simulators. Most
platforms run at 100 or 1000 interrupts per second; the popular x86 PC defaults to
1000, although it used to be 100 in previous versions (up to and including 2.4). As a
general rule, even if you know the value of HZ, you should never count on that specific
value when programming.
It is possible to change the value of HZ for those who want systems with a different
clock interrupt frequency. If you change HZ in the header file, you need to recompile
PAGE 184
the kernel and all modules with the new value. You might want to raise HZ to get a
more fine-grained resolution in your asynchronous tasks, if you are willing to pay the
overhead of the extra timer interrupts to achieve your goals. Actually, raising HZ to
1000 was pretty common with x86 industrial systems using Version 2.4 or 2.2 of the
kernel. With current versions, however, the best approach to the timer interrupt is to
keep the default value for HZ, by virtue of our complete trust in the kernel developers,
who have certainly chosen the best value. Besides, some internal calculations are
currently implemented only for HZ in the range from 12 to 1535 (see <linux/timex.h>
and RFC-1589).
Every time a timer interrupt occurs, the value of an internal kernel counter is incremented.
The counter is initialized to 0 at system boot, so it represents the number of
clock ticks since last boot. The counter is a 64-bit variable (even on 32-bit architectures)
and is called jiffies_64. However, driver writers normally access the jiffies
variable, an unsigned long that is the same as either jiffies_64 or its least significant
bits. Using jiffies is usually preferred because it is faster, and accesses to the 64-bit
jiffies_64 value are not necessarily atomic on all architectures.
In addition to the low-resolution kernel-managed jiffy mechanism, some CPU platforms
feature a high-resolution counter that software can read. Although its actual
use varies somewhat across platforms, it's sometimes a very powerful tool.
Using the jiffies Counter
The counter and the utility functions to read it live in <linux/jiffies.h>, although
you'll usually just include <linux/sched.h>, that automatically pulls jiffies.h in. Needless
to say, both jiffies and jiffies_64 must be considered read-only.
Whenever your code needs to remember the current value of jiffies, it can simply
access the unsigned long variable, which is declared as volatile to tell the compiler
not to optimize memory reads. You need to read the current counter whenever your
code needs to calculate a future time stamp, as shown in the following example:
#include <linux/jiffies.h>
unsigned long j, stamp_1, stamp_half, stamp_n;
j = jiffies; /* read the current value */
stamp_1 = j + HZ; /* 1 second in the future */
stamp_half = j + HZ/2; /* half a second */
stamp_n = j + n * HZ / 1000; /* n milliseconds */
This code has no problem with jiffies wrapping around, as long as different values
are compared in the right way. Even though on 32-bit platforms the counter wraps
around only once every 50 days when HZ is 1000, your code should be prepared to
face that event. To compare your cached value (like stamp_1 above) and the current
value, you should use one of the following macros:
#include <linux/jiffies.h>
int time_after(unsigned long a, unsigned long b);
PAGE 185
int time_before(unsigned long a, unsigned long b);
int time_after_eq(unsigned long a, unsigned long b);
int time_before_eq(unsigned long a, unsigned long b);
The first evaluates true when a, as a snapshot of jiffies, represents a time after b,
the second evaluates true when time a is before time b, and the last two compare for
"after or equal" and "before or equal." The code works by converting the values to
signed long, subtracting them, and comparing the result. If you need to know the difference
between two instances of jiffies in a safe way, you can use the same trick:
diff = (long)t2 - (long)t1;
You can convert a jiffies difference to milliseconds trivially through:
msec = diff * 1000 / HZ;
Sometimes, however, you need to exchange time representations with user space
programs that tend to represent time values with struct timeval and struct timespec.
The two structures represent a precise time quantity with two numbers:
seconds and microseconds are used in the older and popular struct timeval, and seconds
and nanoseconds are used in the newer struct timespec. The kernel exports
four helper functions to convert time values expressed as jiffies to and from those structures:
#include <linux/time.h>
unsigned long timespec_to_jiffies(struct timespec *value);
void jiffies_to_timespec(unsigned long jiffies, struct timespec *value);
unsigned long timeval_to_jiffies(struct timeval *value);
void jiffies_to_timeval(unsigned long jiffies, struct timeval *value);
Accessing the 64-bit jiffy count is not as straightforward as accessing jiffies. While
on 64-bit computer architectures the two variables are actually one, access to the
value is not atomic for 32-bit processors. This means you might read the wrong value
if both halves of the variable get updated while you are reading them. It's extremely
unlikely you'll ever need to read the 64-bit counter, but in case you do, you'll be glad
to know that the kernel exports a specific helper function that does the proper locking
for you:
#include <linux/jiffies.h>
u64 get_jiffies_64(void);
In the above prototype, the u64 type is used. This is one of the types defined by
<linux/types.h>, discussed in Chapter 11, and represents an unsigned 64-bit type.
If you're wondering how 32-bit platforms update both the 32-bit and 64-bit counters
at the same time, read the linker script for your platform (look for a file whose name
matches vmlinux*.lds*). There, the jiffies symbol is defined to access the least significant
word of the 64-bit value, according to whether the platform is little-endian
or big-endian. Actually, the same trick is used for 64-bit platforms, so that the
unsigned long and u64 variables are accessed at the same address.
PAGE 186
Finally, note that the actual clock frequency is almost completely hidden from user
space. The macro HZ always expands to 100 when user-space programs include
param.h, and every counter reported to user space is converted accordingly. This
applies to clock(3), times(2), and any related function. The only evidence available to
users of the HZ value is how fast timer interrupts happen, as shown in /proc/
interrupts. For example, you can obtain HZ by dividing this count by the system
uptime reported in /proc/uptime.
Processor-Specific Registers
If you need to measure very short time intervals or you need extremely high precision
in your figures, you can resort to platform-dependent resources, a choice of precision
over portability.
In modern processors, the pressing demand for empirical performance figures is
thwarted by the intrinsic unpredictability of instruction timing in most CPU designs
due to cache memories, instruction scheduling, and branch prediction. As a
response, CPU manufacturers introduced a way to count clock cycles as an easy and
reliable way to measure time lapses. Therefore, most modern processors include a
counter register that is steadily incremented once at each clock cycle. Nowadays, this
clock counter is the only reliable way to carry out high-resolution timekeeping tasks.
The details differ from platform to platform: the register may or may not be readable
from user space, it may or may not be writable, and it may be 64 or 32 bits wide. In
the last case, you must be prepared to handle overflows just like we did with the jiffy
counter. The register may even not exist for your platform, or it can be implemented
in an external device by the hardware designer, if the CPU lacks the feature and you
are dealing with a special-purpose computer.
Whether or not the register can be zeroed, we strongly discourage resetting it, even
when hardware permits. You might not, after all, be the only user of the counter at
any given time; on some platforms supporting SMP, for example, the kernel depends
on such a counter to be synchronized across processors. Since you can always measure
differences between values, as long as that difference doesn't exceed the overflow
time, you can get the work done without claiming exclusive ownership of the
register by modifying its current value.
The most renowned counter register is the TSC (timestamp counter), introduced in
x86 processors with the Pentium and present in all CPU designs ever since--including
the x86_64 platform. It is a 64-bit register that counts CPU clock cycles; it can be
read from both kernel space and user space.
After including <asm/msr.h> (an x86-specific header whose name stands for
"machine-specific registers"), you can use one of these macros:
rdtsc(low32,high32);
rdtscl(low32);
rdtscll(var64);
PAGE 187
The first macro atomically reads the 64-bit value into two 32-bit variables; the next
one ("read low half") reads the low half of the register into a 32-bit variable, discarding
the high half; the last reads the 64-bit value into a long long variable, hence, the
name. All of these macros store values into their arguments.
Reading the low half of the counter is enough for most common uses of the TSC. A
1-GHz CPU overflows it only once every 4.2 seconds, so you won't need to deal with
multiregister variables if the time lapse you are benchmarking reliably takes less time.
However, as CPU frequencies rise over time and as timing requirements increase,
you'll most likely need to read the 64-bit counter more often in the future.
As an example using only the low half of the register, the following lines measure the
execution of the instruction itself:
unsigned long ini, end;
rdtscl(ini); rdtscl(end);
printk("time lapse: %li\n", end - ini);
Some of the other platforms offer similar functionality, and kernel headers offer an
architecture-independent function that you can use instead of rdtsc. It is called get_cycles,
defined in <asm/timex.h> (included by <linux/timex.h>). Its prototype is:
#include <linux/timex.h>
cycles_t get_cycles(void);
This function is defined for every platform, and it always returns 0 on the platforms
that have no cycle-counter register. The cycles_t type is an appropriate unsigned
type to hold the value read.
Despite the availability of an architecture-independent function, we'd like to take the
opportunity to show an example of inline assembly code. To this aim, we implement
a rdtscl function for MIPS processors that works in the same way as the x86
one.
We base the example on MIPS because most MIPS processors feature a 32-bit
counter as register 9 of their internal "coprocessor 0." To access the register, readable
only from kernel space, you can define the following macro that executes a
"move from coprocessor 0" assembly instruction:*
#define rdtscl(dest) \
__asm__ __volatile__("mfc0 %0,$9; nop" : "=r" (dest))
With this macro in place, the MIPS processor can execute the same code shown earlier
for the x86.
* The trailing nop instruction is required to prevent the compiler from accessing the target register in the
instruction immediately following mfc0. This kind of interlock is typical of RISC processors, and the compiler
can still schedule useful instructions in the delay slots. In this case, we use nop because inline assembly
is a black box for the compiler and no optimization can be performed.
PAGE 188
With gcc inline assembly, the allocation of general-purpose registers is left to the
compiler. The macro just shown uses %0 as a placeholder for "argument 0," which is
later specified as "any register (r) used as output (=)." The macro also states that the
output register must correspond to the C expression dest. The syntax for inline
assembly is very powerful but somewhat complex, especially for architectures that
have constraints on what each register can do (namely, the x86 family). The syntax is
described in the gcc documentation, usually available in the info documentation tree.
The short C-code fragment shown in this section has been run on a K7-class x86 processor
and a MIPS VR4181 (using the macro just described). The former reported a
time lapse of 11 clock ticks and the latter just 2 clock ticks. The small figure was
expected, since RISC processors usually execute one instruction per clock cycle.
There is one other thing worth knowing about timestamp counters: they are not necessarily
synchronized across processors in an SMP system. To be sure of getting a
coherent value, you should disable preemption for code that is querying the counter.
Knowing the Current Time
Kernel code can always retrieve a representation of the current time by looking at the
value of jiffies. Usually, the fact that the value represents only the time since the
last boot is not relevant to the driver, because its life is limited to the system uptime.
As shown, drivers can use the current value of jiffies to calculate time intervals
across events (for example, to tell double-clicks from single-clicks in input device
drivers or calculate timeouts). In short, looking at jiffies is almost always sufficient
when you need to measure time intervals. If you need very precise measurements for
short time lapses, processor-specific registers come to the rescue (although they bring
in serious portability issues).
It's quite unlikely that a driver will ever need to know the wall-clock time, expressed
in months, days, and hours; the information is usually needed only by user programs
such as cron and syslogd. Dealing with real-world time is usually best left to
user space, where the C library offers better support; besides, such code is often too
policy-related to belong in the kernel. There is a kernel function that turns a wallclock
time into a jiffies value, however:
#include <linux/time.h>
unsigned long mktime (unsigned int year, unsigned int mon,
unsigned int day, unsigned int hour,
unsigned int min, unsigned int sec);
To repeat: dealing directly with wall-clock time in a driver is often a sign that policy
is being implemented and should therefore be questioned.
While you won't have to deal with human-readable representations of the time,
sometimes you need to deal with absolute timestamp even in kernel space. To this
aim, <linux/time.h> exports the do_gettimeofday function. When called, it fills a
PAGE 189
struct timeval pointer--the same one used in the gettimeofday system call--with the
familiar seconds and microseconds values. The prototype for do_gettimeofday is:
#include <linux/time.h>
void do_gettimeofday(struct timeval *tv);
The source states that do_gettimeofday has "near microsecond resolution," because it
asks the timing hardware what fraction of the current jiffy has already elapsed. The
precision varies from one architecture to another, however, since it depends on the
actual hardware mechanisms in use. For example, some m68knommu processors,
Sun3 systems, and other m68k systems cannot offer more than jiffy resolution. Pentium
systems, on the other hand, offer very fast and precise subtick measures by
reading the timestamp counter described earlier in this chapter.
The current time is also available (though with jiffy granularity) from the xtime variable,
a struct timespec value. Direct use of this variable is discouraged because it is
difficult to atomically access both the fields. Therefore, the kernel offers the utility
function current_kernel_time:
#include <linux/time.h>
struct timespec current_kernel_time(void);
Code for retrieving the current time in the various ways it is available within the jit
("just in time") module in the source files provided on O'Reilly's FTP site. jit creates
a file called /proc/currentime, which returns the following items in ASCII when read:
- The current jiffies and jiffies_64 values as hex numbers
- The current time as returned by do_gettimeofday
- The timespec returned by current_kernel_time
We chose to use a dynamic /proc file to keep the boilerplate code to a minimum--it's
not worth creating a whole device just to return a little textual information.
The file returns text lines continuously as long as the module is loaded; each read
system call collects and returns one set of data, organized in two lines for better readability.
Whenever you read multiple data sets in less than a timer tick, you'll see the
difference between do_gettimeofday, which queries the hardware, and the other values
that are updated only when the timer ticks.
phon% head -8 /proc/currentime
0x00bdbc1f 0x0000000100bdbc1f 1062370899.630126
1062370899.629161488
0x00bdbc1f 0x0000000100bdbc1f 1062370899.630150
1062370899.629161488
0x00bdbc20 0x0000000100bdbc20 1062370899.630208
1062370899.630161336
0x00bdbc20 0x0000000100bdbc20 1062370899.630233
1062370899.630161336
In the screenshot above, there are two interesting things to note. First, the current_kernel_time value,
though expressed in nanoseconds, has only clock-tick granularity;
PAGE 190
do_gettimeofday consistently reports a later time but not later than the next timer
tick. Second, the 64-bit jiffies counter has the least-significant bit of the upper 32-bit
word set. This happens because the default value for INITIAL_JIFFIES, used at boot
time to initialize the counter, forces a low-word overflow a few minutes after boot
time to help detect problems related to that very overflow. This initial bias in the
counter has no effect, because jiffies is unrelated to wall-clock time. In /proc/
uptime, where the kernel extracts the uptime from the counter, the initial bias is
removed before conversion.
Delaying Execution
Device drivers often need to delay the execution of a particular piece of code for a
period of time, usually to allow the hardware to accomplish some task. In this section
we cover a number of different techniques for achieving delays. The circumstances
of each situation determine which technique is best to use; we go over them
all, and point out the advantages and disadvantages of each.
One important thing to consider is how the delay you need compares with the clock
tick, considering the range of HZ across the various platforms. Delays that are reliably
longer than the clock tick, and don't suffer from its coarse granularity, can make use
of the system clock. Very short delays typically must be implemented with software
loops. In between these two cases lies a gray area. In this chapter, we use the phrase
"long" delay to refer to a multiple-jiffy delay, which can be as low as a few milliseconds
on some platforms, but is still long as seen by the CPU and the kernel.
The following sections talk about the different delays by taking a somewhat long
path from various intuitive but inappropriate solutions to the right solution. We
chose this path because it allows a more in-depth discussion of kernel issues related
to timing. If you are eager to find the right code, just skim through the section.
Long Delays
Occasionally a driver needs to delay execution for relatively long periods--more than
one clock tick. There are a few ways of accomplishing this sort of delay; we start with
the simplest technique, then proceed to the more advanced techniques.
Busy waiting
If you want to delay execution by a multiple of the clock tick, allowing some slack in
the value, the easiest (though not recommended) implementation is a loop that monitors
the jiffy counter. The busy-waiting implementation usually looks like the following
code, where j1 is the value of jiffies at the expiration of the delay:
while (time_before(jiffies, j1))
cpu_relax( );
PAGE 191
The call to cpu_relax invokes an architecture-specific way of saying that you're not
doing much with the processor at the moment. On many systems it does nothing at
all; on symmetric multithreaded ("hyperthreaded") systems, it may yield the core to
the other thread. In any case, this approach should definitely be avoided whenever
possible. We show it here because on occasion you might want to run this code to
better understand the internals of other code.
So let's look at how this code works. The loop is guaranteed to work because jiffies
is declared as volatile by the kernel headers and, therefore, is fetched from memory
any time some C code accesses it. Although technically correct (in that it works as
designed), this busy loop severely degrades system performance. If you didn't configure
your kernel for preemptive operation, the loop completely locks the processor for
the duration of the delay; the scheduler never preempts a process that is running in
kernel space, and the computer looks completely dead until time j1 is reached. The
problem is less serious if you are running a preemptive kernel, because, unless the
code is holding a lock, some of the processor's time can be recovered for other uses.
Busy waits are still expensive on preemptive systems, however.
Still worse, if interrupts happen to be disabled when you enter the loop, jiffies
won't be updated, and the while condition remains true forever. Running a preemptive
kernel won't help either, and you'll be forced to hit the big red button.
This implementation of delaying code is available, like the following ones, in the jit
module. The /proc/jit* files created by the module delay a whole second each time
you read a line of text, and lines are guaranteed to be 20 bytes each. If you want to
test the busy-wait code, you can read /proc/jitbusy, which busy-loops for one second
for each line it returns.
Be sure to read, at most, one line (or a few lines) at a time from /proc/
jitbusy. The simplified kernel mechanism to register /proc files invokes
the read method over and over to fill the data buffer the user
requested. Therefore, a command such as cat /proc/jitbusy, if it reads 4
KB at a time, freezes the computer for 205 seconds.
The suggested command to read /proc/jitbusy is dd bs=20 < /proc/jitbusy, optionally
specifying the number of blocks as well. Each 20-byte line returned by the file represents
the value the jiffy counter had before and after the delay. This is a sample run
on an otherwise unloaded computer:
phon% dd bs=20 count=5 < /proc/jitbusy
1686518 1687518
1687519 1688519
1688520 1689520
1689520 1690520
1690521 1691521
PAGE 192
All looks good: delays are exactly one second (1000 jiffies), and the next read system
call starts immediately after the previous one is over. But let's see what happens on a
system with a large number of CPU-intensive processes running (and nonpreemptive
kernel):
phon% dd bs=20 count=5 < /proc/jitbusy
1911226 1912226
1913323 1914323
1919529 1920529
1925632 1926632
1931835 1932835
Here, each read system call delays exactly one second, but the kernel can take more
than 5 seconds before scheduling the dd process so it can issue the next system call.
That's expected in a multitasking system; CPU time is shared between all running
processes, and a CPU-intensive process has its dynamic priority reduced. (A discussion
of scheduling policies is outside the scope of this book.)
The test under load shown above has been performed while running the load50 sample
program. This program forks a number of processes that do nothing, but do it in
a CPU-intensive way. The program is part of the sample files accompanying this
book, and forks 50 processes by default, although the number can be specified on
the command line. In this chapter, and elsewhere in the book, the tests with a loaded
system have been performed with load50 running in an otherwise idle computer.
If you repeat the command while running a preemptible kernel, you'll find no noticeable
difference on an otherwise idle CPU and the following behavior under load:
phon% dd bs=20 count=5 < /proc/jitbusy
14940680 14942777
14942778 14945430
14945431 14948491
14948492 14951960
14951961 14955840
Here, there is no significant delay between the end of a system call and the beginning
of the next one, but the individual delays are far longer than one second: up to
3.8 seconds in the example shown and increasing over time. These values demonstrate
that the process has been interrupted during its delay, scheduling other processes.
The gap between system calls is not the only scheduling option for this
process, so no special delay can be seen there.
Yielding the processor
As we have seen, busy waiting imposes a heavy load on the system as a whole; we
would like to find a better technique. The first change that comes to mind is to
PAGE 193
explicitly release the CPU when we're not interested in it. This is accomplished by
calling the schedule function, declared in <linux/sched.h>:
while (time_before(jiffies, j1)) {
schedule( );
}
This loop can be tested by reading /proc/jitsched as we read /proc/jitbusy above. However,
is still isn't optimal. The current process does nothing but release the CPU, but
it remains in the run queue. If it is the only runnable process, it actually runs (it calls
the scheduler, which selects the same process, which calls the scheduler, which...).
In other words, the load of the machine (the average number of running processes) is
at least one, and the idle task (process number 0, also called swapper for historical
reasons) never runs. Though this issue may seem irrelevant, running the idle task
when the computer is idle relieves the processor's workload, decreasing its temperature
and increasing its lifetime, as well as the duration of the batteries if the computer
happens to be your laptop. Moreover, since the process is actually executing
during the delay, it is accountable for all the time it consumes.
The behavior of /proc/jitsched is actually similar to running /proc/jitbusy under a preemptive
kernel. This is a sample run, on an unloaded system:
phon% dd bs=20 count=5 < /proc/jitsched
1760205 1761207
1761209 1762211
1762212 1763212
1763213 1764213
1764214 1765217
It's interesting to note that each read sometimes ends up waiting a few clock ticks
more than requested. This problem gets worse and worse as the system gets busy,
and the driver could end up waiting longer than expected. Once a process releases
the processor with schedule, there are no guarantees that the process will get the processor
back anytime soon. Therefore, calling schedule in this manner is not a safe
solution to the driver's needs, in addition to being bad for the computing system as a
whole. If you test jitsched while running load50, you can see that the delay associated
to each line is extended by a few seconds, because other processes are using the
CPU when the timeout expires.
Timeouts
The suboptimal delay loops shown up to now work by watching the jiffy counter
without telling anyone. But the best way to implement a delay, as you may imagine,
is usually to ask the kernel to do it for you. There are two ways of setting up jiffy-based
timeouts, depending on whether your driver is waiting for other events or not.
PAGE 194
If your driver uses a wait queue to wait for some other event, but you also want to be
sure that it runs within a certain period of time, it can use wait_event_timeout or
wait_event_interruptible_timeout:
#include <linux/wait.h>
long wait_event_timeout(wait_queue_head_t q, condition, long timeout);
long wait_event_interruptible_timeout(wait_queue_head_t q,
condition, long timeout);
These functions sleep on the given wait queue, but they return after the timeout
(expressed in jiffies) expires. Thus, they implement a bounded sleep that does not go
on forever. Note that the timeout value represents the number of jiffies to wait, not
an absolute time value. The value is represented by a signed number, because it
sometimes is the result of a subtraction, although the functions complain through a
printk statement if the provided timeout is negative. If the timeout expires, the functions
return 0; if the process is awakened by another event, it returns the remaining
delay expressed in jiffies. The return value is never negative, even if the delay is
greater than expected because of system load.
The /proc/jitqueue file shows a delay based on wait_event_interruptible_timeout,
although the module has no event to wait for, and uses 0 as a condition:
wait_queue_head_t wait;
init_waitqueue_head (&wait);
wait_event_interruptible_timeout(wait, 0, delay);
The observed behaviour, when reading /proc/jitqueue, is nearly optimal, even under
load:
phon% dd bs=20 count=5 < /proc/jitqueue
2027024 2028024
2028025 2029025
2029026 2030026
2030027 2031027
2031028 2032028
Since the reading process (dd above) is not in the run queue while waiting for the
timeout, you see no difference in behavior whether the code is run in a preemptive
kernel or not.
wait_event_timeout and wait_event_interruptible_timeout were designed with a hardware
driver in mind, where execution could be resumed in either of two ways: either
somebody calls wake_up on the wait queue, or the timeout expires. This doesn't
apply to jitqueue, as nobody ever calls wake_up on the wait queue (after all, no other
code even knows about it), so the process always wakes up when the timeout
expires. To accommodate for this very situation, where you want to delay execution
waiting for no specific event, the kernel offers the schedule_timeout function so you
can avoid declaring and using a superfluous wait queue head:
#include <linux/sched.h>
signed long schedule_timeout(signed long timeout);
PAGE 195
Here, timeout is the number of jiffies to delay. The return value is 0 unless the function
returns before the given timeout has elapsed (in response to a signal). schedule_timeout
requires that the caller first set the current process state, so a typical call looks like:
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout (delay);
The previous lines (from /proc/jitschedto) cause the process to sleep until the given
time has passed. Since wait_event_interruptible_timeout relies on schedule_timeout
internally, we won't bother showing the numbers jitschedto returns, because they are
the same as those of jitqueue. Once again, it is worth noting that an extra time interval
could pass between the expiration of the timeout and when your process is actually
scheduled to execute.
In the example just shown, the first line calls set_current_state to set things up so that
the scheduler won't run the current process again until the timeout places it back in
TASK_RUNNING state. To achieve an uninterruptible delay, use TASK_UNINTERRUPTIBLE
instead. If you forget to change the state of the current process, a call to schedule_
timeout behaves like a call to schedule (i.e., the jitsched behavior), setting up a timer
that is not used.
If you want to play with the four jit files under different system situations or different
kernels, or try other ways to delay execution, you may want to configure the
amount of the delay when loading the module by setting the delay module parameter.
Short Delays
When a device driver needs to deal with latencies in its hardware, the delays involved
are usually a few dozen microseconds at most. In this case, relying on the clock tick
is definitely not the way to go.
The kernel functions ndelay, udelay, and mdelay serve well for short delays, delaying
execution for the specified number of nanoseconds, microseconds, or milliseconds
respectively.* Their prototypes are:
#include <linux/delay.h>
void ndelay(unsigned long nsecs);
void udelay(unsigned long usecs);
void mdelay(unsigned long msecs);
The actual implementations of the functions are in <asm/delay.h>, being architecture-specific,
and sometimes build on an external function. Every architecture implements
udelay, but the other functions may or may not be defined; if they are not,
<linux/delay.h> offers a default version based on udelay. In all cases, the delay
achieved is at least the requested value but could be more; actually, no platform currently
achieves nanosecond precision, although several ones offer submicrosecond
* The u in udelay represents the Greek letter mu and stands for micro.
PAGE 196
precision. Delaying more than the requested value is usually not a problem, as short
delays in a driver are usually needed to wait for the hardware, and the requirements
are to wait for at least a given time lapse.
The implementation of udelay (and possibly ndelay too) uses a software loop based on
the processor speed calculated at boot time, using the integer variable loops_per_jiffy.
If you want to look at the actual code, however, be aware that the x86 implementation
is quite a complex one because of the different timing sources it uses, based on what
CPU type is running the code.
To avoid integer overflows in loop calculations, udelay and ndelay impose an upper
bound in the value passed to them. If your module fails to load and displays an unresolved
symbol, __bad_udelay, it means you called udelay with too large an argument.
Note, however, that the compile-time check can be performed only on
constant values and that not all platforms implement it. As a general rule, if you are
trying to delay for thousands of nanoseconds, you should be using udelay rather than
ndelay; similarly, millisecond-scale delays should be done with mdelay and not one
of the finer-grained functions.
It's important to remember that the three delay functions are busy-waiting; other
tasks can't be run during the time lapse. Thus, they replicate, though on a different
scale, the behavior of jitbusy. Thus, these functions should only be used when there
is no practical alternative.
There is another way of achieving millisecond (and longer) delays that does not
involve busy waiting. The file <linux/delay.h> declares these functions:
void msleep(unsigned int millisecs);
unsigned long msleep_interruptible(unsigned int millisecs);
void ssleep(unsigned int seconds)
The first two functions puts the calling process to sleep for the given number of
millisecs. A call to msleep is uninterruptible; you can be sure that the process sleeps
for at least the given number of milliseconds. If your driver is sitting on a wait queue
and you want a wakeup to break the sleep, use msleep_interruptible. The return value
from msleep_interruptible is normally 0; if, however, the process is awakened early,
the return value is the number of milliseconds remaining in the originally requested
sleep period. A call to ssleep puts the process into an uninterruptible sleep for the
given number of seconds.
In general, if you can tolerate longer delays than requested, you should use
schedule_timeout, msleep, or ssleep.
Kernel Timers
Whenever you need to schedule an action to happen later, without blocking the current
process until that time arrives, kernel timers are the tool for you. These timers
PAGE 197
are used to schedule execution of a function at a particular time in the future, based
on the clock tick, and can be used for a variety of tasks; for example, polling a device
by checking its state at regular intervals when the hardware can't fire interrupts.
Other typical uses of kernel timers are turning off the floppy motor or finishing
another lengthy shut down operation. In such cases, delaying the return from close
would impose an unnecessary (and surprising) cost on the application program.
Finally, the kernel itself uses the timers in several situations, including the implementation
of schedule_timeout.
A kernel timer is a data structure that instructs the kernel to execute a user-defined
function with a user-defined argument at a user-defined time. The implementation
resides in <linux/timer.h> and kernel/timer.c and is described in detail in the section
"The Implementation of Kernel Timers."
The functions scheduled to run almost certainly do not run while the process that
registered them is executing. They are, instead, run asynchronously. Until now,
everything we have done in our sample drivers has run in the context of a process
executing system calls. When a timer runs, however, the process that scheduled it
could be asleep, executing on a different processor, or quite possibly has exited
altogether.
This asynchronous execution resembles what happens when a hardware interrupt
happens (which is discussed in detail in Chapter 10). In fact, kernel timers are run as
the result of a "software interrupt." When running in this sort of atomic context,
your code is subject to a number of constraints. Timer functions must be atomic in
all the ways we discussed in the section "Spinlocks and Atomic Context" in
Chapter 1, but there are some additional issues brought about by the lack of a process
context. We will introduce these constraints now; they will be seen again in several
places in later chapters. Repetition is called for because the rules for atomic
contexts must be followed assiduously, or the system will find itself in deep trouble.
A number of actions require the context of a process in order to be executed. When
you are outside of process context (i.e., in interrupt context), you must observe the
following rules:
- No access to user space is allowed. Because there is no process context, there is
no path to the user space associated with any particular process.
- The current pointer is not meaningful in atomic mode and cannot be used since
the relevant code has no connection with the process that has been interrupted.
- No sleeping or scheduling may be performed. Atomic code may not call schedule or
a form of wait_event, nor may it call any other function that could sleep.
For example, calling kmalloc(..., GFP_KERNEL) is against the rules. Semaphores
also must not be used since they can sleep.
PAGE 198
Kernel code can tell if it is running in interrupt context by calling the function in_interrupt( ),
which takes no parameters and returns nonzero if the processor is currently
running in interrupt context, either hardware interrupt or software interrupt.
A function related to in_interrupt( ) is in_atomic( ). Its return value is nonzero whenever
scheduling is not allowed; this includes hardware and software interrupt contexts
as well as any time when a spinlock is held. In the latter case, current may be valid, but
access to user space is forbidden, since it can cause scheduling to happen. Whenever
you are using in_interrupt( ), you should really consider whether in_atomic( ) is what
you actually mean. Both functions are declared in <asm/hardirq.h>
One other important feature of kernel timers is that a task can reregister itself to run
again at a later time. This is possible because each timer_list structure is unlinked
from the list of active timers before being run and can, therefore, be immediately relinked
elsewhere. Although rescheduling the same task over and over might appear
to be a pointless operation, it is sometimes useful. For example, it can be used to
implement the polling of devices.
It's also worth knowing that in an SMP system, the timer function is executed by the
same CPU that registered it, to achieve better cache locality whenever possible.
Therefore, a timer that reregisters itself always runs on the same CPU.
An important feature of timers that should not be forgotten, though, is that they are
a potential source of race conditions, even on uniprocessor systems. This is a direct
result of their being asynchronous with other code. Therefore, any data structures
accessed by the timer function should be protected from concurrent access, either by
being atomic types (discussed in the section "Atomic Variables" in Chapter 1) or by
using spinlocks (discussed in Chapter 5).
The Timer API
The kernel provides drivers with a number of functions to declare, register, and
remove kernel timers. The following excerpt shows the basic building blocks:
#include <linux/timer.h>
struct timer_list {
/* ... */
unsigned long expires;
void (*function)(unsigned long);
unsigned long data;
};
void init_timer(struct timer_list *timer);
struct timer_list TIMER_INITIALIZER(_function, _expires, _data);
void add_timer(struct timer_list * timer);
int del_timer(struct timer_list * timer);
PAGE 199
The data structure includes more fields than the ones shown, but those three are the
ones that are meant to be accessed from outside the timer code iteslf. The expires
field represents the jiffies value when the timer is expected to run; at that time, the
function function is called with data as an argument. If you need to pass multiple
items in the argument, you can bundle them as a single data structure and pass a
pointer cast to unsigned long, a safe practice on all supported architectures and
pretty common in memory management (as discussed in Chapter 15). The expires
value is not a jiffies_64 item because timers are not expected to expire very far in
the future, and 64-bit operations are slow on 32-bit platforms.
The structure must be initialized before use. This step ensures that all the fields are
properly set up, including the ones that are opaque to the caller. Initialization can be
performed by calling init_timer or assigning TIMER_INITIALIZER to a static structure,
according to your needs. After initialization, you can change the three public fields
before calling add_timer. To disable a registered timer before it expires, call del_timer.
The jit module includes a sample file, /proc/jitimer (for "just in timer"), that returns
one header line and six data lines. The data lines represent the current environment
where the code is running; the first one is generated by the read file operation and
the others by a timer. The following output was recorded while compiling a kernel:
phon% cat /proc/jitimer
time delta inirq pid cpu command
33565837 0 0 1269 0 cat
33565847 10 1 1271 0 sh
33565857 10 1 1273 0 cpp0
33565867 10 1 1273 0 cpp0
33565877 10 1 1274 0 cc1
33565887 10 1 1274 0 cc1
In this output, the time field is the value of jiffies when the code runs, delta is the
change in jiffies since the previous line, inirq is the Boolean value returned by in_interrupt,
pid and command refer to the current process, and cpu is the number of the
CPU being used (always 0 on uniprocessor systems).
If you read /proc/jitimer while the system is unloaded, you'll find that the context of
the timer is process 0, the idle task, which is called "swapper" mainly for historical
reasons.
The timer used to generate /proc/jitimer data is run every 10 jiffies by default, but you
can change the value by setting the tdelay (timer delay) parameter when loading the
module.
The following code excerpt shows the part of jit related to the jitimer timer. When a
process attempts to read our file, we set up the timer as follows:
unsigned long j = jiffies;
/* fill the data for our timer function */
data->prevjiffies = j;
PAGE 200
data->buf = buf2;
data->loops = JIT_ASYNC_LOOPS;
/* register the timer */
data->timer.data = (unsigned long)data;
data->timer.function = jit_timer_fn;
data->timer.expires = j + tdelay; /* parameter */
add_timer(&data->timer);
/* wait for the buffer to fill */
wait_event_interruptible(data->wait, !data->loops);
The actual timer function looks like this:
void jit_timer_fn(unsigned long arg)
{
struct jit_data *data = (struct jit_data *)arg;
unsigned long j = jiffies;
data->buf += sprintf(data->buf, "%9li %3li %i %6i %i %s\n",
j, j - data->prevjiffies, in_interrupt( ) ? 1 : 0,
current->pid, smp_processor_id( ), current->comm);
if (--data->loops) {
data->timer.expires += tdelay;
data->prevjiffies = j;
add_timer(&data->timer);
} else {
wake_up_interruptible(&data->wait);
}
}
The timer API includes a few more functions than the ones introduced above. The
following set completes the list of kernel offerings:
int mod_timer(struct timer_list *timer, unsigned long expires);
Updates the expiration time of a timer, a common task for which a timeout
timer is used (again, the motor-off floppy timer is a typical example). mod_timer
can be called on inactive timers as well, where you normally use add_timer.
int del_timer_sync(struct timer_list *timer);
Works like del_timer, but also guarantees that when it returns, the timer function
is not running on any CPU. del_timer_sync is used to avoid race conditions
on SMP systems and is the same as del_timer in UP kernels. This function should
be preferred over del_timer in most situations. This function can sleep if it is
called from a nonatomic context but busy waits in other situations. Be very careful
about calling del_timer_sync while holding locks; if the timer function
attempts to obtain the same lock, the system can deadlock. If the timer function
reregisters itself, the caller must first ensure that this reregistration will not happen;
this is usually accomplished by setting a "shutting down" flag, which is
checked by the timer function.
PAGE 201
int timer_pending(const struct timer_list * timer);
Returns true or false to indicate whether the timer is currently scheduled to run
by reading one of the opaque fields of the structure.
The Implementation of Kernel Timers
Although you won't need to know how kernel timers are implemented in order to
use them, the implementation is interesting, and a look at its internals is worthwhile.
The implementation of the timers has been designed to meet the following requirements
and assumptions:
- Timer management must be as lightweight as possible.
- The design should scale well as the number of active timers increases.
- Most timers expire within a few seconds or minutes at most, while timers with
long delays are pretty rare.
- A timer should run on the same CPU that registered it.
The solution devised by kernel developers is based on a per-CPU data structure. The
timer_list structure includes a pointer to that data structure in its base field. If base is
NULL, the timer is not scheduled to run; otherwise, the pointer tells which data structure
(and, therefore, which CPU) runs it. Per-CPU data items are described in the
section "Per-CPU Variables" in Chapter 8.
Whenever kernel code registers a timer (via add_timer or mod_timer), the operation
is eventually performed by internal_add_timer (in kernel/timer.c) which, in turn,
adds the new timer to a double-linked list of timers within a "cascading table" associated
to the current CPU.
The cascading table works like this: if the timer expires in the next 0 to 255 jiffies, it
is added to one of the 256 lists devoted to short-range timers using the least significant
bits of the expires field. If it expires farther in the future (but before 16,384 jiffies),
it is added to one of 64 lists based on bits 9-14 of the expires field. For timers
expiring even farther, the same trick is used for bits 15-20, 21-26, and 27-31. Timers
with an expire field pointing still farther in the future (something that can happen
only on 64-bit platforms) are hashed with a delay value of 0xffffffff, and
timers with expires in the past are scheduled to run at the next timer tick. (A timer
that is already expired may sometimes be registered in high-load situations, especially
if you run a preemptible kernel.)
When __run_timers is fired, it executes all pending timers for the current timer tick.
If jiffies is currently a multiple of 256, the function also rehashes one of the next level
lists of timers into the 256 short-term lists, possibly cascading one or more of
the other levels as well, according to the bit representation of jiffies.
PAGE 202
This approach, while exceedingly complex at first sight, performs very well both with
few timers and with a large number of them. The time required to manage each
active timer is independent of the number of timers already registered and is limited
to a few logic operations on the binary representation of its expires field. The only
cost associated with this implementation is the memory for the 512 list heads (256
short-term lists and 4 groups of 64 more lists)--i.e., 4 KB of storage.
The function __run_timers, as shown by /proc/jitimer, is run in atomic context. In
addition to the limitations we already described, this brings in an interesting feature:
the timer expires at just the right time, even if you are not running a preemptible kernel,
and the CPU is busy in kernel space. You can see what happens when you read
/proc/jitbusy in the background and /proc/jitimer in the foreground. Although the system
appears to be locked solid by the busy-waiting system call, the kernel timers still
work fine.
Keep in mind, however, that a kernel timer is far from perfect, as it suffers from jitter
and other artifacts induced by hardware interrupts, as well as other timers and other
asynchronous tasks. While a timer associated with simple digital I/O can be enough
for simple tasks like running a stepper motor or other amateur electronics, it is usually
not suitable for production systems in industrial environments. For such tasks,
you'll most likely need to resort to a real-time kernel extension.
Tasklets
Another kernel facility related to timing issues is the tasklet mechanism. It is mostly
used in interrupt management (we'll see it again in Chapter 10.)
Tasklets resemble kernel timers in some ways. They are always run at interrupt time,
they always run on the same CPU that schedules them, and they receive an unsigned long
argument. Unlike kernel timers, however, you can't ask to execute the function
at a specific time. By scheduling a tasklet, you simply ask for it to be executed at a
later time chosen by the kernel. This behavior is especially useful with interrupt handlers,
where the hardware interrupt must be managed as quickly as possible, but
most of the data management can be safely delayed to a later time. Actually, a
tasklet, just like a kernel timer, is executed (in atomic mode) in the context of a "soft
interrupt," a kernel mechanism that executes asynchronous tasks with hardware
interrupts enabled.
A tasklet exists as a data structure that must be initialized before use. Initialization
can be performed by calling a specific function or by declaring the structure using
certain macros:
#include <linux/interrupt.h>
struct tasklet_struct {
/* ... */
PAGE 203
void (*func)(unsigned long);
unsigned long data;
};
void tasklet_init(struct tasklet_struct *t,
void (*func)(unsigned long), unsigned long data);
DECLARE_TASKLET(name, func, data);
DECLARE_TASKLET_DISABLED(name, func, data);
Tasklets offer a number of interesting features:
- A tasklet can be disabled and re-enabled later; it won't be executed until it is
enabled as many times as it has been disabled.
- Just like timers, a tasklet can reregister itself.
- A tasklet can be scheduled to execute at normal priority or high priority. The latter group is always executed first.
- Tasklets may be run immediately if the system is not under heavy load but never
later than the next timer tick.
- A tasklets can be concurrent with other tasklets but is strictly serialized with
respect to itself--the same tasklet never runs simultaneously on more than one
processor. Also, as already noted, a tasklet always runs on the same CPU that
schedules it.
The jit module includes two files, /proc/jitasklet and /proc/jitasklethi, that return the
same data as /proc/jitimer, introduced in the section "Kernel Timers." When you
read one of the files, you get back a header and six data lines. The first data line
describes the context of the calling process, and the other lines describe the context
of successive runs of a tasklet procedure. This is a sample run while compiling a kernel:
phon% cat /proc/jitasklet
time delta inirq pid cpu command
6076139 0 0 4370 0 cat
6076140 1 1 4368 0 cc1
6076141 1 1 4368 0 cc1
6076141 0 1 2 0 ksoftirqd/0
6076141 0 1 2 0 ksoftirqd/0
6076141 0 1 2 0 ksoftirqd/0
As confirmed by the above data, the tasklet is run at the next timer tick as long as the
CPU is busy running a process, but it is run immediately when the CPU is otherwise
idle. The kernel provides a set of ksoftirqd kernel threads, one per CPU, just to run
"soft interrupt" handlers, such as the tasklet_action function. Thus, the final three
runs of the tasklet take place in the context of the ksoftirqd kernel thread associated
to CPU 0. The jitasklethi implementation uses a high-priority tasklet, explained in an
upcoming list of functions.
The actual code in jit that implements /proc/jitasklet and /proc/jitasklethi is almost
identical to the code that implements /proc/jitimer, but it uses the tasklet calls instead
PAGE 204
of the timer ones. The following list lays out in detail the kernel interface to tasklets
after the tasklet structure has been initialized:
void tasklet_disable(struct tasklet_struct *t);
This function disables the given tasklet. The tasklet may still be scheduled with
tasklet_schedule, but its execution is deferred until the tasklet has been enabled
again. If the tasklet is currently running, this function busy-waits until the tasklet
exits; thus, after calling tasklet_disable, you can be sure that the tasklet is not
running anywhere in the system.
void tasklet_disable_nosync(struct tasklet_struct *t);
Disable the tasklet, but without waiting for any currently-running function to
exit. When it returns, the tasklet is disabled and won't be scheduled in the future
until re-enabled, but it may be still running on another CPU when the function
returns.
void tasklet_enable(struct tasklet_struct *t);
Enables a tasklet that had been previously disabled. If the tasklet has already
been scheduled, it will run soon. A call to tasklet_enable must match each call to
tasklet_disable, as the kernel keeps track of the "disable count" for each tasklet.
void tasklet_schedule(struct tasklet_struct *t);
Schedule the tasklet for execution. If a tasklet is scheduled again before it has a
chance to run, it runs only once. However, if it is scheduled while it runs, it runs
again after it completes; this ensures that events occurring while other events are
being processed receive due attention. This behavior also allows a tasklet to
reschedule itself.
void tasklet_hi_schedule(struct tasklet_struct *t);
Schedule the tasklet for execution with higher priority. When the soft interrupt
handler runs, it deals with high-priority tasklets before other soft interrupt tasks,
including "normal" tasklets. Ideally, only tasks with low-latency requirements
(such as filling the audio buffer) should use this function, to avoid the additional
latencies introduced by other soft interrupt handlers. Actually, /proc/
jitasklethi shows no human-visible difference from /proc/jitasklet.
void tasklet_kill(struct tasklet_struct *t);
This function ensures that the tasklet is not scheduled to run again; it is usually
called when a device is being closed or the module removed. If the tasklet is
scheduled to run, the function waits until it has executed. If the tasklet reschedules
itself, you must prevent it from rescheduling itself before calling tasklet_kill,
as with del_timer_sync.
Tasklets are implemented in kernel/softirq.c. The two tasklet lists (normal and high priority)
are declared as per-CPU data structures, using the same CPU-affinity mechanism
used by kernel timers. The data structure used in tasklet management is a simple
linked list, because tasklets have none of the sorting requirements of kernel
timers.
PAGE 205
Workqueues
Workqueues are, superficially, similar to tasklets; they allow kernel code to request
that a function be called at some future time. There are, however, some significant
differences between the two, including:
- Tasklets run in software interrupt context with the result that all tasklet code
must be atomic. Instead, workqueue functions run in the context of a special
kernel process; as a result, they have more flexibility. In particular, workqueue
functions can sleep.
- Tasklets always run on the processor from which they were originally submitted.
Work-queues work in the same way, by default.
- Kernel code can request that the execution of workqueue functions be delayed
for an explicit interval.
The key difference between the two is that tasklets execute quickly, for a short period
of time, and in atomic mode, while workqueue functions may have higher latency but
need not be atomic. Each mechanism has situations where it is appropriate.
Workqueues have a type of struct workqueue_struct, which is defined in <linux/
workqueue.h>. A workqueue must be explicitly created before use, using one of the
following two functions:
struct workqueue_struct *create_workqueue(const char *name);
struct workqueue_struct *create_singlethread_workqueue(const char *name);
Each workqueue has one or more dedicated processes ("kernel threads"), which run
functions submitted to the queue. If you use create_workqueue, you get a workqueue
that has a dedicated thread for each processor on the system. In many cases,
all those threads are simply overkill; if a single worker thread will suffice, create the
workqueue with create_singlethread_workqueue instead.
To submit a task to a workqueue, you need to fill in a work_struct structure. This
can be done at compile time as follows:
DECLARE_WORK(name, void (*function)(void *), void *data);
Where name is the name of the structure to be declared, function is the function that is
to be called from the workqueue, and data is a value to pass to that function. If you
need to set up the work_struct structure at runtime, use the following two macros:
INIT_WORK(struct work_struct *work, void (*function)(void *), void *data);
PREPARE_WORK(struct work_struct *work, void (*function)(void *), void *data);
INIT_WORK does a more thorough job of initializing the structure; you should use
it the first time that structure is set up. PREPARE_WORK does almost the same job,
but it does not initialize the pointers used to link the work_struct structure into the
workqueue. If there is any possibility that the structure may currently be submitted
PAGE 206
to a workqueue, and you need to change that structure, use PREPARE_WORK
rather than INIT_WORK.
There are two functions for submitting work to a workqueue:
int queue_work(struct workqueue_struct *queue, struct work_struct *work);
int queue_delayed_work(struct workqueue_struct *queue,
struct work_struct *work, unsigned long delay);
Either one adds work to the given queue. If queue_delayed_work is used, however, the
actual work is not performed until at least delay jiffies have passed. The return value
from these functions is 0 if the work was successfully added to the queue; a nonzero
result means that this work_struct structure was already waiting in the queue, and
was not added a second time.
At some time in the future, the work function will be called with the given data
value. The function will be running in the context of the worker thread, so it can
sleep if need be--although you should be aware of how that sleep might affect any
other tasks submitted to the same workqueue. What the function cannot do, however,
is access user space. Since it is running inside a kernel thread, there simply is no
user space to access.
Should you need to cancel a pending workqueue entry, you may call:
int cancel_delayed_work(struct work_struct *work);
The return value is nonzero if the entry was canceled before it began execution. The
kernel guarantees that execution of the given entry will not be initiated after a call to
cancel_delayed_work. If cancel_delayed_work returns 0, however, the entry may have
already been running on a different processor, and might still be running after a call
to cancel_delayed_work. To be absolutely sure that the work function is not running
anywhere in the system after cancel_delayed_work returns 0, you must follow that
call with a call to:
void flush_workqueue(struct workqueue_struct *queue);
After flush_workqueue returns, no work function submitted prior to the call is running
anywhere in the system.
When you are done with a workqueue, you can get rid of it with:
void destroy_workqueue(struct workqueue_struct *queue);
The Shared Queue
A device driver, in many cases, does not need its own workqueue. If you only submit
tasks to the queue occasionally, it may be more efficient to simply use the shared,
default workqueue that is provided by the kernel. If you use this queue, however,
you must be aware that you will be sharing it with others. Among other things, that
means that you should not monopolize the queue for long periods of time (no long
sleeps), and it may take longer for your tasks to get their turn in the processor.
PAGE 207
The jiq ("just in queue") module exports two files that demonstrate the use of the
shared workqueue. They use a single work_struct structure, which is set up this way:
static struct work_struct jiq_work;
/* this line is in jiq_init( ) */
INIT_WORK(&jiq_work, jiq_print_wq, &jiq_data);
When a process reads /proc/jiqwq, the module initiates a series of trips through the
shared workqueue with no delay. The function it uses is:
int schedule_work(struct work_struct *work);
Note that a different function is used when working with the shared queue; it
requires only the work_struct structure for an argument. The actual code in jiq looks
like this:
prepare_to_wait(&jiq_wait, &wait, TASK_INTERRUPTIBLE);
schedule_work(&jiq_work);
schedule( );
finish_wait(&jiq_wait, &wait);
The actual work function prints out a line just like the jit module does, then, if need
be, resubmits the work_struct structure into the workqueue. Here is jiq_print_wq in
its entirety:
static void jiq_print_wq(void *ptr)
{
struct clientdata *data = (struct clientdata *) ptr;
if (! jiq_print (ptr))
return;
if (data->delay)
schedule_delayed_work(&jiq_work, data->delay);
else
schedule_work(&jiq_work);
}
If the user is reading the delayed device (/proc/jiqwqdelay), the work function resubmits
itself in the delayed mode with schedule_delayed_work:
int schedule_delayed_work(struct work_struct *work, unsigned long delay);
If you look at the output from these two devices, it looks something like:
% cat /proc/jiqwq
time delta preempt pid cpu command
1113043 0 0 7 1 events/1
1113043 0 0 7 1 events/1
1113043 0 0 7 1 events/1
1113043 0 0 7 1 events/1
1113043 0 0 7 1 events/1
% cat /proc/jiqwqdelay
time delta preempt pid cpu command
1122066 1 0 6 0 events/0
PAGE 208
1122067 1 0 6 0 events/0
1122068 1 0 6 0 events/0
1122069 1 0 6 0 events/0
1122070 1 0 6 0 events/0
When /proc/jiqwq is read, there is no obvious delay between the printing of each line.
When, instead, /proc/jiqwqdelay is read, there is a delay of exactly one jiffy between
each line. In either case, we see the same process name printed; it is the name of the
kernel thread that implements the shared workqueue. The CPU number is printed
after the slash; we never know which CPU will be running when the /proc file is read,
but the work function will always run on the same processor thereafter.
If you need to cancel a work entry submitted to the shared queue, you may use
cancel_delayed_work, as described above. Flushing the shared workqueue requires a
separate function, however:
void flush_scheduled_work(void);
Since you do not know who else might be using this queue, you never really know
how long it might take for flush_scheduled_work to return.
Quick Reference
This chapter introduced the following symbols.
Timekeeping
#include <linux/param.h>
HZ
The HZ symbol specifies the number of clock ticks generated per second.
#include <linux/jiffies.h>
volatile unsigned long jiffies
u64 jiffies_64
The jiffies_64 variable is incremented once for each clock tick; thus, it's incremented
HZ times per second. Kernel code most often refers to jiffies, which is
the same as jiffies_64 on 64-bit platforms and the least significant half of it on
32-bit platforms.
int time_after(unsigned long a, unsigned long b);
int time_before(unsigned long a, unsigned long b);
int time_after_eq(unsigned long a, unsigned long b);
int time_before_eq(unsigned long a, unsigned long b);
These Boolean expressions compare jiffies in a safe way, without problems in
case of counter overflow and without the need to access jiffies_64.
u64 get_jiffies_64(void);
Retrieves jiffies_64 without race conditions.
PAGE 209
#include <linux/time.h>
unsigned long timespec_to_jiffies(struct timespec *value);
void jiffies_to_timespec(unsigned long jiffies, struct timespec *value);
unsigned long timeval_to_jiffies(struct timeval *value);
void jiffies_to_timeval(unsigned long jiffies, struct timeval *value);
Converts time representations between jiffies and other representations.
#include <asm/msr.h>
rdtsc(low32,high32);
rdtscl(low32);
rdtscll(var32);
x86-specific macros to read the timestamp counter. They read it as two 32-bit
halves, read only the lower half, or read all of it into a long long variable.
#include <linux/timex.h>
cycles_t get_cycles(void);
Returns the timestamp counter in a platform-independent way. If the CPU offers
no timestamp feature, 0 is returned.
#include <linux/time.h>
unsigned long mktime(year, mon, day, h, m, s);
Returns the number of seconds since the Epoch, based on the six unsigned int
arguments.
void do_gettimeofday(struct timeval *tv);
Returns the current time, as seconds and microseconds since the Epoch, with the
best resolution the hardware can offer. On most platforms the resolution is one
microsecond or better, although some platforms offer only jiffies resolution.
struct timespec current_kernel_time(void);
Returns the current time with the resolution of one jiffy.
Delays
#include <linux/wait.h>
long wait_event_interruptible_timeout(wait_queue_head_t *q, condition, signed long timeout);
Puts the current process to sleep on the wait queue, installing a timeout value
expressed in jiffies. Use schedule_timeout (below) for noninterruptible sleeps.
#include <linux/sched.h>
signed long schedule_timeout(signed long timeout);
Calls the scheduler after ensuring that the current process is awakened at timeout
expiration. The caller must invoke set_current_state first to put itself in an
interruptible or noninterruptible sleep state.
PAGE 210
#include <linux/delay.h>
void ndelay(unsigned long nsecs);
void udelay(unsigned long usecs);
void mdelay(unsigned long msecs);
Introduces delays of an integer number of nanoseconds, microseconds, and milliseconds.
The delay achieved is at least the requested value, but it can be more.
The argument to each function must not exceed a platform-specific limit (usually
a few thousands).
void msleep(unsigned int millisecs);
unsigned long msleep_interruptible(unsigned int millisecs);
void ssleep(unsigned int seconds);
Puts the process to sleep for the given number of milliseconds (or seconds, in the
case of ssleep).
Kernel Timers
#include <asm/hardirq.h>
int in_interrupt(void);
int in_atomic(void);
Returns a Boolean value telling whether the calling code is executing in interrupt
context or atomic context. Interrupt context is outside of a process context,
either during hardware or software interrupt processing. Atomic context is
when you can't schedule either an interrupt context or a process's context with a
spinlock held.
#include <linux/timer.h>
void init_timer(struct timer_list * timer);
struct timer_list TIMER_INITIALIZER(_function, _expires, _data);
This function and the static declaration of the timer structure are the two ways
to initialize a timer_list data structure.
void add_timer(struct timer_list * timer);
Registers the timer structure to run on the current CPU.
int mod_timer(struct timer_list *timer, unsigned long expires);
Changes the expiration time of an already scheduled timer structure. It can also
act as an alternative to add_timer.
int timer_pending(struct timer_list * timer);
Macro that returns a Boolean value stating whether the timer structure is already
registered to run.
void del_timer(struct timer_list * timer);
void del_timer_sync(struct timer_list * timer);
Removes a timer from the list of active timers. The latter function ensures that
the timer is not currently running on another CPU.
PAGE 211
Tasklets
#include <linux/interrupt.h>
DECLARE_TASKLET(name, func, data);
DECLARE_TASKLET_DISABLED(name, func, data);
void tasklet_init(struct tasklet_struct *t, void (*func)(unsigned long), unsigned long data);
The first two macros declare a tasklet structure, while the tasklet_init function
initializes a tasklet structure that has been obtained by allocation or other
means. The second DECLARE macro marks the tasklet as disabled.
void tasklet_disable(struct tasklet_struct *t);
void tasklet_disable_nosync(struct tasklet_struct *t);
void tasklet_enable(struct tasklet_struct *t);
Disables and reenables a tasklet. Each disable must be matched with an enable
(you can disable the tasklet even if it's already disabled). The function tasklet_
disable waits for the tasklet to terminate if it is running on another CPU. The
nosync version doesn't take this extra step.
void tasklet_schedule(struct tasklet_struct *t);
void tasklet_hi_schedule(struct tasklet_struct *t);
Schedules a tasklet to run, either as a "normal" tasklet or a high-priority one.
When soft interrupts are executed, high-priority tasklets are dealt with first,
while normal tasklets run last.
void tasklet_kill(struct tasklet_struct *t);
Removes the tasklet from the list of active ones, if it's scheduled to run. Like
tasklet_disable, the function may block on SMP systems waiting for the tasklet to
terminate if it's currently running on another CPU.
Workqueues
#include <linux/workqueue.h>
struct workqueue_struct;
struct work_struct;
The structures representing a workqueue and a work entry, respectively.
struct workqueue_struct *create_workqueue(const char *name);
struct workqueue_struct *create_singlethread_workqueue(const char *name);
void destroy_workqueue(struct workqueue_struct *queue);
Functions for creating and destroying workqueues. A call to create_workqueue
creates a queue with a worker thread on each processor in the system;
instead, create_singlethread_workqueue creates a workqueue with a single
worker process.
PAGE 212
DECLARE_WORK(name, void (*function)(void *), void *data);
INIT_WORK(struct work_struct *work, void (*function)(void *), void *data);
PREPARE_WORK(struct work_struct *work, void (*function)(void *), void *data);
Macros that declare and initialize workqueue entries.
int queue_work(struct workqueue_struct *queue, struct work_struct *work);
int queue_delayed_work(struct workqueue_struct *queue, struct work_struct *work, unsigned long delay);
Functions that queue work for execution from a workqueue.
int cancel_delayed_work(struct work_struct *work);
void flush_workqueue(struct workqueue_struct *queue);
Use cancel_delayed_work to remove an entry from a workqueue; flush_workqueue
ensures that no workqueue entries are running anywhere in the system.
int schedule_work(struct work_struct *work);
int schedule_delayed_work(struct work_struct *work, unsigned long delay);
void flush_scheduled_work(void);
Functions for working with the shared workqueue.
PAGE 213
CHAPTER 8
Allocating Memory
Thus far, we have used kmalloc and kfree for the allocation and freeing of memory.
The Linux kernel offers a richer set of memory allocation primitives, however. In this
chapter, we look at other ways of using memory in device drivers and how to optimize
your system's memory resources. We do not get into how the different architectures
actually administer memory. Modules are not involved in issues of
segmentation, paging, and so on, since the kernel offers a unified memory management
interface to the drivers. In addition, we won't describe the internal details of
memory management in this chapter, but defer it to Chapter 15.
The Real Story of kmalloc
The kmalloc allocation engine is a powerful tool and easily learned because of its
similarity to malloc. The function is fast (unless it blocks) and doesn't clear the memory
it obtains; the allocated region still holds its previous content.* The allocated
region is also contiguous in physical memory. In the next few sections, we talk in
detail about kmalloc, so you can compare it with the memory allocation techniques
that we discuss later.
The Flags Argument
Remember that the prototype for kmalloc is:
#include <linux/slab.h>
void *kmalloc(size_t size, int flags);
* Among other things, this implies that you should explicitly clear any memory that might be exposed to user
space or written to a device; otherwise, you risk disclosing information that should be kept private.
PAGE 214
The first argument to kmalloc is the size of the block to be allocated. The second
argument, the allocation flags, is much more interesting, because it controls the
behavior of kmalloc in a number of ways.
The most commonly used flag, GFP_KERNEL, means that the allocation (internally performed
by calling, eventually, __get_free_pages, which is the source of the GFP_ prefix)
is performed on behalf of a process running in kernel space. In other words, this
means that the calling function is executing a system call on behalf of a process.
Using GFP_KERNEL means that kmalloc can put the current process to sleep waiting for
a page when called in low-memory situations. A function that allocates memory
using GFP_KERNEL must, therefore, be reentrant and cannot be running in atomic context.
While the current process sleeps, the kernel takes proper action to locate some
free memory, either by flushing buffers to disk or by swapping out memory from a
user process.
GFP_KERNEL isn't always the right allocation flag to use; sometimes kmalloc is called
from outside a process's context. This type of call can happen, for instance, in interrupt
handlers, tasklets, and kernel timers. In this case, the current process should
not be put to sleep, and the driver should use a flag of GFP_ATOMIC instead. The kernel
normally tries to keep some free pages around in order to fulfill atomic allocation.
When GFP_ATOMIC is used, kmalloc can use even the last free page. If that last
page does not exist, however, the allocation fails.
Other flags can be used in place of or in addition to GFP_KERNEL and GFP_ATOMIC,
although those two cover most of the needs of device drivers. All the flags are defined
in <linux/gfp.h>, and individual flags are prefixed with a double underscore, such
as __GFP_DMA. In addition, there are symbols that represent frequently used combinations
of flags; these lack the prefix and are sometimes called allocation priorities. The
latter include:
GFP_ATOMIC
Used to allocate memory from interrupt handlers and other code outside of a
process context. Never sleeps.
GFP_KERNEL
Normal allocation of kernel memory. May sleep.
GFP_USER
Used to allocate memory for user-space pages; it may sleep.
GFP_HIGHUSER
Like GFP_USER, but allocates from high memory, if any. High memory is
described in the next subsection.
GFP_NOIO
GFP_NOFS
These flags function like GFP_KERNEL, but they add restrictions on what the kernel
can do to satisfy the request. A GFP_NOFS allocation is not allowed to perform
PAGE 215
any filesystem calls, while GFP_NOIO disallows the initiation of any I/O at all.
They are used primarily in the filesystem and virtual memory code where an allocation
may be allowed to sleep, but recursive filesystem calls would be a bad
idea.
The allocation flags listed above can be augmented by an ORing in any of the following
flags, which change how the allocation is carried out:
__GFP_DMA
This flag requests allocation to happen in the DMA-capable memory zone. The
exact meaning is platform-dependent and is explained in the following section.
__GFP_HIGHMEM
This flag indicates that the allocated memory may be located in high memory.
__GFP_COLD
Normally, the memory allocator tries to return "cache warm" pages--pages that
are likely to be found in the processor cache. Instead, this flag requests a "cold"
page, which has not been used in some time. It is useful for allocating pages for
DMA reads, where presence in the processor cache is not useful. See the section
"Direct Memory Access" in Chapter 1 for a full discussion of how to allocate
DMA buffers.
__GFP_NOWARN
This rarely used flag prevents the kernel from issuing warnings (with printk)
when an allocation cannot be satisfied.
__GFP_HIGH
This flag marks a high-priority request, which is allowed to consume even the
last pages of memory set aside by the kernel for emergencies.
__GFP_REPEAT
__GFP_NOFAIL
__GFP_NORETRY
These flags modify how the allocator behaves when it has difficulty satisfying an
allocation. __GFP_REPEAT means "try a little harder" by repeating the attempt-but
the allocation can still fail. The __GFP_NOFAIL flag tells the allocator never to
fail; it works as hard as needed to satisfy the request. Use of __GFP_NOFAIL is very
strongly discouraged; there will probably never be a valid reason to use it in a
device driver. Finally, __GFP_NORETRY tells the allocator to give up immediately if
the requested memory is not available.
Memory zones
Both __GFP_DMA and __GFP_HIGHMEM have a platform-dependent role, although their
use is valid for all platforms.
The Linux kernel knows about a minimum of three memory zones: DMA-capable
memory, normal memory, and high memory. While allocation normally happens in
PAGE 216
the normal zone, setting either of the bits just mentioned requires memory to be allocated
from a different zone. The idea is that every computer platform that must
know about special memory ranges (instead of considering all RAM equivalents) will
fall into this abstraction.
DMA-capable memory is memory that lives in a preferential address range, where
peripherals can perform DMA access. On most sane platforms, all memory lives in
this zone. On the x86, the DMA zone is used for the first 16 MB of RAM, where legacy
ISA devices can perform DMA; PCI devices have no such limit.
High memory is a mechanism used to allow access to (relatively) large amounts of
memory on 32-bit platforms. This memory cannot be directly accessed from the kernel
without first setting up a special mapping and is generally harder to work with. If
your driver uses large amounts of memory, however, it will work better on large systems
if it can use high memory. See the section "High and Low Memory" in
Chapter 1 for a detailed description of how high memory works and how to use it.
Whenever a new page is allocated to fulfill a memory allocation request, the kernel
builds a list of zones that can be used in the search. If __GFP_DMA is specified, only the
DMA zone is searched: if no memory is available at low addresses, allocation fails. If no
special flag is present, both normal and DMA memory are searched; if __GFP_HIGHMEM is
set, all three zones are used to search a free page. (Note, however, that kmalloc cannot
allocate high memory.)
The situation is more complicated on nonuniform memory access (NUMA) systems.
As a general rule, the allocator attempts to locate memory local to the processor performing
the allocation, although there are ways of changing that behavior.
The mechanism behind memory zones is implemented in mm/page_alloc.c, while initialization
of the zone resides in platform-specific files, usually in mm/init.c within
the arch tree. We'll revisit these topics in Chapter 15.
The Size Argument
The kernel manages the system's physical memory, which is available only in pagesized
chunks. As a result, kmalloc looks rather different from a typical user-space
malloc implementation. A simple, heap-oriented allocation technique would quickly
run into trouble; it would have a hard time working around the page boundaries.
Thus, the kernel uses a special page-oriented allocation technique to get the best use
from the system's RAM.
Linux handles memory allocation by creating a set of pools of memory objects of
fixed sizes. Allocation requests are handled by going to a pool that holds sufficiently
large objects and handing an entire memory chunk back to the requester. The memory
management scheme is quite complex, and the details of it are not normally all
that interesting to device driver writers.
PAGE 217
The one thing driver developers should keep in mind, though, is that the kernel can
allocate only certain predefined, fixed-size byte arrays. If you ask for an arbitrary
amount of memory, you're likely to get slightly more than you asked for, up to twice
as much. Also, programmers should remember that the smallest allocation that
kmalloc can handle is as big as 32 or 64 bytes, depending on the page size used by
the system's architecture.
There is an upper limit to the size of memory chunks that can be allocated by kmalloc.
That limit varies depending on architecture and kernel configuration options. If
your code is to be completely portable, it cannot count on being able to allocate anything
larger than 128 KB. If you need more than a few kilobytes, however, there are
better ways than kmalloc to obtain memory, which we describe later in this chapter.
Lookaside Caches
A device driver often ends up allocating many objects of the same size, over and over.
Given that the kernel already maintains a set of memory pools of objects that are all
the same size, why not add some special pools for these high-volume objects? In fact,
the kernel does implement a facility to create this sort of pool, which is often called a
lookaside cache. Device drivers normally do not exhibit the sort of memory behavior
that justifies using a lookaside cache, but there can be exceptions; the USB and SCSI
drivers in Linux 2.6 use caches.
The cache manager in the Linux kernel is sometimes called the "slab allocator." For
that reason, its functions and types are declared in <linux/slab.h>. The slab allocator
implements caches that have a type of kmem_cache_t; they are created with a call to
kmem_cache_create:
kmem_cache_t *kmem_cache_create(const char *name, size_t size,
size_t offset,
unsigned long flags,
void (*constructor)(void *, kmem_cache_t *,
unsigned long flags),
void (*destructor)(void *, kmem_cache_t *,
unsigned long flags));
The function creates a new cache object that can host any number of memory areas
all of the same size, specified by the size argument. The name argument is associated
with this cache and functions as housekeeping information usable in tracking problems;
usually, it is set to the name of the type of structure that is cached. The cache
keeps a pointer to the name, rather than copying it, so the driver should pass in a
pointer to a name in static storage (usually the name is just a literal string). The name
cannot contain blanks.
The offset is the offset of the first object in the page; it can be used to ensure a particular
alignment for the allocated objects, but you most likely will use 0 to request
PAGE 218
the default value. flags controls how allocation is done and is a bit mask of the following
flags:
SLAB_NO_REAP
Setting this flag protects the cache from being reduced when the system is looking
for memory. Setting this flag is normally a bad idea; it is important to avoid
restricting the memory allocator's freedom of action unnecessarily.
SLAB_HWCACHE_ALIGN
This flag requires each data object to be aligned to a cache line; actual alignment
depends on the cache layout of the host platform. This option can be a good
choice if your cache contains items that are frequently accessed on SMP
machines. The padding required to achieve cache line alignment can end up
wasting significant amounts of memory, however.
SLAB_CACHE_DMA
This flag requires each data object to be allocated in the DMA memory zone.
There is also a set of flags that can be used during the debugging of cache allocations;
see mm/slab.c for the details. Usually, however, these flags are set globally via a
kernel configuration option on systems used for development.
The constructor and destructor arguments to the function are optional functions
(but there can be no destructor without a constructor); the former can be used to initialize
newly allocated objects, and the latter can be used to "clean up" objects prior
to their memory being released back to the system as a whole.
Constructors and destructors can be useful, but there are a few constraints that you
should keep in mind. A constructor is called when the memory for a set of objects is
allocated; because that memory may hold several objects, the constructor may be
called multiple times. You cannot assume that the constructor will be called as an
immediate effect of allocating an object. Similarly, destructors can be called at some
unknown future time, not immediately after an object has been freed. Constructors
and destructors may or may not be allowed to sleep, according to whether they are
passed the SLAB_CTOR_ATOMIC flag (where CTOR is short for constructor).
For convenience, a programmer can use the same function for both the constructor
and destructor; the slab allocator always passes the SLAB_CTOR_CONSTRUCTOR flag when
the callee is a constructor.
Once a cache of objects is created, you can allocate objects from it by calling
kmem_cache_alloc:
void *kmem_cache_alloc(kmem_cache_t *cache, int flags);
Here, the cache argument is the cache you have created previously; the flags are the
same as you would pass to kmalloc and are consulted if kmem_cache_alloc needs to
go out and allocate more memory itself.
To free an object, use kmem_cache_free:
void kmem_cache_free(kmem_cache_t *cache, const void *obj);
PAGE 219
When driver code is finished with the cache, typically when the module is unloaded,
it should free its cache as follows:
int kmem_cache_destroy(kmem_cache_t *cache);
The destroy operation succeeds only if all objects allocated from the cache have
been returned to it. Therefore, a module should check the return status from
kmem_cache_destroy; a failure indicates some sort of memory leak within the module
(since some of the objects have been dropped).
One side benefit to using lookaside caches is that the kernel maintains statistics on
cache usage. These statistics may be obtained from /proc/slabinfo.
A scull Based on the Slab Caches: scullc
Time for an example. scullc is a cut-down version of the scull module that implements
only the bare device--the persistent memory region. Unlike scull, which uses
kmalloc, scullc uses memory caches. The size of the quantum can be modified at
compile time and at load time, but not at runtime--that would require creating a
new memory cache, and we didn't want to deal with these unneeded details.
scullc is a complete example that can be used to try out the slab allocator. It differs
from scull only in a few lines of code. First, we must declare our own slab cache:
/* declare one cache pointer: use it for all devices */
kmem_cache_t *scullc_cache;
The creation of the slab cache is handled (at module load time) in this way:
/* scullc_init: create a cache for our quanta */
scullc_cache = kmem_cache_create("scullc", scullc_quantum,
0, SLAB_HWCACHE_ALIGN, NULL, NULL); /* no ctor/dtor */
if (!scullc_cache) {
scullc_cleanup( );
return -ENOMEM;
}
This is how it allocates memory quanta:
/* Allocate a quantum using the memory cache */
if (!dptr->data[s_pos]) {
dptr->data[s_pos] = kmem_cache_alloc(scullc_cache, GFP_KERNEL);
if (!dptr->data[s_pos])
goto nomem;
memset(dptr->data[s_pos], 0, scullc_quantum);
}
And these lines release memory:
for (i = 0; i < qset; i++)
if (dptr->data[i])
kmem_cache_free(scullc_cache, dptr->data[i]);
PAGE 220
Finally, at module unload time, we have to return the cache to the system:
/* scullc_cleanup: release the cache of our quanta */
if (scullc_cache)
kmem_cache_destroy(scullc_cache);
The main differences in passing from scull to scullc are a slight speed improvement
and better memory use. Since quanta are allocated from a pool of memory fragments
of exactly the right size, their placement in memory is as dense as possible, as
opposed to scull quanta, which bring in an unpredictable memory fragmentation.
Memory Pools
There are places in the kernel where memory allocations cannot be allowed to fail.
As a way of guaranteeing allocations in those situations, the kernel developers created
an abstraction known as a memory pool (or "mempool"). A memory pool is
really just a form of a lookaside cache that tries to always keep a list of free memory
around for use in emergencies.
A memory pool has a type of mempool_t (defined in <linux/mempool.h>); you can create
one with mempool_create:
mempool_t *mempool_create(int min_nr,
mempool_alloc_t *alloc_fn,
mempool_free_t *free_fn,
void *pool_data);
The min_nr argument is the minimum number of allocated objects that the pool
should always keep around. The actual allocation and freeing of objects is handled
by alloc_fn and free_fn, which have these prototypes:
typedef void *(mempool_alloc_t)(int gfp_mask, void *pool_data);
typedef void (mempool_free_t)(void *element, void *pool_data);
The final parameter to mempool_create (pool_data) is passed to alloc_fn and free_fn.
If need be, you can write special-purpose functions to handle memory allocations for
mempools. Usually, however, you just want to let the kernel slab allocator handle that
task for you. There are two functions (mempool_alloc_slab and mempool_free_slab)
that perform the impedance matching between the memory pool allocation prototypes
and kmem_cache_alloc and kmem_cache_free. Thus, code that sets up memory
pools often looks like the following:
cache = kmem_cache_create(. . .);
pool = mempool_create(MY_POOL_MINIMUM,
mempool_alloc_slab, mempool_free_slab,
cache);
Once the pool has been created, objects can be allocated and freed with:
void *mempool_alloc(mempool_t *pool, int gfp_mask);
void mempool_free(void *element, mempool_t *pool);
PAGE 221
When the mempool is created, the allocation function will be called enough times to
create a pool of preallocated objects. Thereafter, calls to mempool_alloc attempt to
acquire additional objects from the allocation function; should that allocation fail,
one of the preallocated objects (if any remain) is returned. When an object is freed
with mempool_free, it is kept in the pool if the number of preallocated objects is currently
below the minimum; otherwise, it is to be returned to the system.
A mempool can be resized with:
int mempool_resize(mempool_t *pool, int new_min_nr, int gfp_mask);
This call, if successful, resizes the pool to have at least new_min_nr objects.
If you no longer need a memory pool, return it to the system with:
void mempool_destroy(mempool_t *pool);
You must return all allocated objects before destroying the mempool, or a kernel
oops results.
If you are considering using a mempool in your driver, please keep one thing in
mind: mempools allocate a chunk of memory that sits in a list, idle and unavailable
for any real use. It is easy to consume a great deal of memory with mempools. In
almost every case, the preferred alternative is to do without the mempool and simply
deal with the possibility of allocation failures instead. If there is any way for your
driver to respond to an allocation failure in a way that does not endanger the integrity
of the system, do things that way. Use of mempools in driver code should be
rare.
get_free_page and Friends
If a module needs to allocate big chunks of memory, it is usually better to use a pageoriented
technique. Requesting whole pages also has other advantages, which are
introduced in Chapter 15.
To allocate pages, the following functions are available:
get_zeroed_page(unsigned int flags);
Returns a pointer to a new page and fills the page with zeros.
__get_free_page(unsigned int flags);
Similar to get_zeroed_page, but doesn't clear the page.
__get_free_pages(unsigned int flags, unsigned int order);
Allocates and returns a pointer to the first byte of a memory area that is potentially
several (physically contiguous) pages long but doesn't zero the area.
The flags argument works in the same way as with kmalloc; usually either GFP_KERNEL
or GFP_ATOMIC is used, perhaps with the addition of the __GFP_DMA flag (for memory
that can be used for ISA direct-memory-access operations) or __GFP_HIGHMEM when
PAGE 222
high memory can be used.* order is the base-two logarithm of the number of pages
you are requesting or freeing (i.e., log2N). For example, order is 0 if you want one
page and 3 if you request eight pages. If order is too big (no contiguous area of that
size is available), the page allocation fails. The get_order function, which takes an integer
argument, can be used to extract the order from a size (that must be a power of
two) for the hosting platform. The maximum allowed value for order is 10 or 11 (corresponding
to 1024 or 2048 pages), depending on the architecture. The chances of an
order-10 allocation succeeding on anything other than a freshly booted system with a
lot of memory are small, however.
If you are curious, /proc/buddyinfo tells you how many blocks of each order are available
for each memory zone on the system.
When a program is done with the pages, it can free them with one of the following
functions. The first function is a macro that falls back on the second:
void free_page(unsigned long addr);
void free_pages(unsigned long addr, unsigned long order);
If you try to free a different number of pages from what you allocated, the memory
map becomes corrupted, and the system gets in trouble at a later time.
It's worth stressing that __get_free_pages and the other functions can be called at any
time, subject to the same rules we saw for kmalloc. The functions can fail to allocate
memory in certain circumstances, particularly when GFP_ATOMIC is used. Therefore,
the program calling these allocation functions must be prepared to handle an allocation
failure.
Although kmalloc(GFP_KERNEL) sometimes fails when there is no available memory,
the kernel does its best to fulfill allocation requests. Therefore, it's easy to degrade
system responsiveness by allocating too much memory. For example, you can bring
the computer down by pushing too much data into a scull device; the system starts
crawling while it tries to swap out as much as possible in order to fulfill the kmalloc
request. Since every resource is being sucked up by the growing device, the computer
is soon rendered unusable; at that point, you can no longer even start a new
process to try to deal with the problem. We don't address this issue in scull, since it
is just a sample module and not a real tool to put into a multiuser system. As a programmer,
you must be careful nonetheless, because a module is privileged code and
can open new security holes in the system (the most likely is a denial-of-service hole
like the one just outlined).
* Although alloc_pages (described shortly) should really be used for allocating high-memory pages,
for reasons we can't really get into until Chapter 15.
PAGE 223
A scull Using Whole Pages: scullp
In order to test page allocation for real, we have released the scullp module together
with other sample code. It is a reduced scull, just like scullc introduced earlier.
Memory quanta allocated by scullp are whole pages or page sets: the scullp_order
variable defaults to 0 but can be changed at either compile or load time.
The following lines show how it allocates memory:
/* Here's the allocation of a single quantum */
if (!dptr->data[s_pos]) {
dptr->data[s_pos] =
(void *)__get_free_pages(GFP_KERNEL, dptr->order);
if (!dptr->data[s_pos])
goto nomem;
memset(dptr->data[s_pos], 0, PAGE_SIZE << dptr->order);
}
The code to deallocate memory in scullp looks like this:
/* This code frees a whole quantum-set */
for (i = 0; i < qset; i++)
if (dptr->data[i])
free_pages((unsigned long)(dptr->data[i]),
dptr->order);
At the user level, the perceived difference is primarily a speed improvement and better
memory use, because there is no internal fragmentation of memory. We ran some
tests copying 4 MB from scull0 to scull1 and then from scullp0 to scullp1; the results
showed a slight improvement in kernel-space processor usage.
The performance improvement is not dramatic, because kmalloc is designed to be
fast. The main advantage of page-level allocation isn't actually speed, but rather
more efficient memory usage. Allocating by pages wastes no memory, whereas using
kmalloc wastes an unpredictable amount of memory because of allocation granularity.
But the biggest advantage of the __get_free_page functions is that the pages obtained
are completely yours, and you could, in theory, assemble the pages into a linear area
by appropriate tweaking of the page tables. For example, you can allow a user process
to mmap memory areas obtained as single unrelated pages. We discuss this kind
of operation in Chapter 15, where we show how scullp offers memory mapping,
something that scull cannot offer.
The alloc_pages Interface
For completeness, we introduce another interface for memory allocation, even
though we will not be prepared to use it until after Chapter 15. For now, suffice it to
say that struct page is an internal kernel structure that describes a page of memory.
As we will see, there are many places in the kernel where it is necessary to work with
PAGE 224
page structures; they are especially useful in any situation where you might be dealing
with high memory, which does not have a constant address in kernel space.
The real core of the Linux page allocator is a function called alloc_pages_node:
struct page *alloc_pages_node(int nid, unsigned int flags,
unsigned int order);
This function also has two variants (which are simply macros); these are the versions
that you will most likely use:
struct page *alloc_pages(unsigned int flags, unsigned int order);
struct page *alloc_page(unsigned int flags);
The core function, alloc_pages_node, takes three arguments. nid is the NUMA node
ID* whose memory should be allocated, flags is the usual GFP_ allocation flags, and
order is the size of the allocation. The return value is a pointer to the first of (possibly
many) page structures describing the allocated memory, or, as usual, NULL on failure.
alloc_pages simplifies the situation by allocating the memory on the current NUMA
node (it calls alloc_pages_node with the return value from numa_node_id as the nid
parameter). And, of course, alloc_page omits the order parameter and allocates a single
page.
To release pages allocated in this manner, you should use one of the following:
void __free_page(struct page *page);
void __free_pages(struct page *page, unsigned int order);
void free_hot_page(struct page *page);
void free_cold_page(struct page *page);
If you have specific knowledge of whether a single page's contents are likely to be
resident in the processor cache, you should communicate that to the kernel with
free_hot_page (for cache-resident pages) or free_cold_page. This information helps
the memory allocator optimize its use of memory across the system.
vmalloc and Friends
The next memory allocation function that we show you is vmalloc, which allocates a
contiguous memory region in the virtual address space. Although the pages are not consecutive
in physical memory (each page is retrieved with a separate call to alloc_page),
the kernel sees them as a contiguous range of addresses. vmalloc returns 0 (the NULL
address) if an error occurs, otherwise, it returns a pointer to a linear memory area of size
at least size.
* NUMA (nonuniform memory access) computers are multiprocessor systems where memory is "local" to
specific groups of processors ("nodes"). Access to local memory is faster than access to nonlocal memory.
On such systems, allocating memory on the correct node is important. Driver authors do not normally have
to worry about NUMA issues, however.
PAGE 225
We describe vmalloc here because it is one of the fundamental Linux memory allocation
mechanisms. We should note, however, that use of vmalloc is discouraged in
most situations. Memory obtained from vmalloc is slightly less efficient to work with,
and, on some architectures, the amount of address space set aside for vmalloc is relatively
small. Code that uses vmalloc is likely to get a chilly reception if submitted for
inclusion in the kernel. If possible, you should work directly with individual pages
rather than trying to smooth things over with vmalloc.
That said, let's see how vmalloc works. The prototypes of the function and its relatives
(ioremap, which is not strictly an allocation function, is discussed later in this
section) are as follows:
#include <linux/vmalloc.h>
void *vmalloc(unsigned long size);
void vfree(void * addr);
void *ioremap(unsigned long offset, unsigned long size);
void iounmap(void * addr);
It's worth stressing that memory addresses returned by kmalloc and _get_free_pages
are also virtual addresses. Their actual value is still massaged by the MMU (the memory
management unit, usually part of the CPU) before it is used to address physical
memory.* vmalloc is not different in how it uses the hardware, but rather in how the
kernel performs the allocation task.
The (virtual) address range used by kmalloc and __get_free_pages features a one-toone
mapping to physical memory, possibly shifted by a constant PAGE_OFFSET value;
the functions don't need to modify the page tables for that address range. The
address range used by vmalloc and ioremap, on the other hand, is completely synthetic,
and each allocation builds the (virtual) memory area by suitably setting up the
page tables.
This difference can be perceived by comparing the pointers returned by the allocation
functions. On some platforms (for example, the x86), addresses returned by
vmalloc are just beyond the addresses that kmalloc uses. On other platforms (for
example, MIPS, IA-64, and x86_64), they belong to a completely different address
range. Addresses available for vmalloc are in the range from VMALLOC_START to
VMALLOC_END. Both symbols are defined in <asm/pgtable.h>.
Addresses allocated by vmalloc can't be used outside of the microprocessor, because
they make sense only on top of the processor's MMU. When a driver needs a real
physical address (such as a DMA address, used by peripheral hardware to drive the
system's bus), you can't easily use vmalloc. The right time to call vmalloc is when
* Actually, some architectures define ranges of "virtual" addresses as reserved to address physical memory.
When this happens, the Linux kernel takes advantage of the feature, and both the kernel and __get_free_pages
addresses lie in one of those memory ranges. The difference is transparent to device drivers and other code
that is not directly involved with the memory-management kernel subsystem.
PAGE 226
you are allocating memory for a large sequential buffer that exists only in software.
It's important to note that vmalloc has more overhead than __get_free_pages,
because it must both retrieve the memory and build the page tables. Therefore, it
doesn't make sense to call vmalloc to allocate just one page.
An example of a function in the kernel that uses vmalloc is the create_module system
call, which uses vmalloc to get space for the module being created. Code and data of
the module are later copied to the allocated space using copy_from_user. In this way,
the module appears to be loaded into contiguous memory. You can verify, by looking
in /proc/kallsyms, that kernel symbols exported by modules lie in a different
memory range from symbols exported by the kernel proper.
Memory allocated with vmalloc is released by vfree, in the same way that kfree
releases memory allocated by kmalloc.
Like vmalloc, ioremap builds new page tables; unlike vmalloc, however, it doesn't
actually allocate any memory. The return value of ioremap is a special virtual address
that can be used to access the specified physical address range; the virtual address
obtained is eventually released by calling iounmap.
ioremap is most useful for mapping the (physical) address of a PCI buffer to (virtual)
kernel space. For example, it can be used to access the frame buffer of a PCI video
device; such buffers are usually mapped at high physical addresses, outside of the
address range for which the kernel builds page tables at boot time. PCI issues are
explained in more detail in Chapter 12.
It's worth noting that for the sake of portability, you should not directly access
addresses returned by ioremap as if they were pointers to memory. Rather, you
should always use readb and the other I/O functions introduced in Chapter 9. This
requirement applies because some platforms, such as the Alpha, are unable to
directly map PCI memory regions to the processor address space because of differences
between PCI specs and Alpha processors in how data is transferred.
Both ioremap and vmalloc are page oriented (they work by modifying the page
tables); consequently, the relocated or allocated size is rounded up to the nearest
page boundary. ioremap simulates an unaligned mapping by "rounding down" the
address to be remapped and by returning an offset into the first remapped page.
One minor drawback of vmalloc is that it can't be used in atomic context because,
internally, it uses kmalloc(GFP_KERNEL) to acquire storage for the page tables, and
therefore could sleep. This shouldn't be a problem--if the use of __get_free_page isn't
good enough for an interrupt handler, the software design needs some cleaning up.
PAGE 227
A scull Using Virtual Addresses: scullv
Sample code using vmalloc is provided in the scullv module. Like scullp, this module
is a stripped-down version of scull that uses a different allocation function to obtain
space for the device to store data.
The module allocates memory 16 pages at a time. The allocation is done in large
chunks to achieve better performance than scullp and to show something that takes
too long with other allocation techniques to be feasible. Allocating more than one
page with __get_free_pages is failure prone, and even when it succeeds, it can be
slow. As we saw earlier, vmalloc is faster than other functions in allocating several
pages, but somewhat slower when retrieving a single page, because of the overhead
of page-table building. scullv is designed like scullp. order specifies the "order" of
each allocation and defaults to 4. The only difference between scullv and scullp is in
allocation management. These lines use vmalloc to obtain new memory:
/* Allocate a quantum using virtual addresses */
if (!dptr->data[s_pos]) {
dptr->data[s_pos] =
(void *)vmalloc(PAGE_SIZE << dptr->order);
if (!dptr->data[s_pos])
goto nomem;
memset(dptr->data[s_pos], 0, PAGE_SIZE << dptr->order);
}
and these lines release memory:
/* Release the quantum-set */
for (i = 0; i < qset; i++)
if (dptr->data[i])
vfree(dptr->data[i]);
If you compile both modules with debugging enabled, you can look at their data
allocation by reading the files they create in /proc. This snapshot was taken on an
x86_64 system:
salma% cat /tmp/bigfile > /dev/scullp0; head -5 /proc/scullpmem
Device 0: qset 500, order 0, sz 1535135
item at 000001001847da58, qset at 000001001db4c000
0:1001db56000
1:1003d1c7000
salma% cat /tmp/bigfile > /dev/scullv0; head -5 /proc/scullvmem
Device 0: qset 500, order 4, sz 1535135
item at 000001001847da58, qset at 0000010013dea000
0:ffffff0001177000
1:ffffff0001188000
PAGE 228
The following output, instead, came from an x86 system:
rudo% cat /tmp/bigfile > /dev/scullp0; head -5 /proc/scullpmem
Device 0: qset 500, order 0, sz 1535135
item at ccf80e00, qset at cf7b9800
0:ccc58000
1:cccdd000
rudo% cat /tmp/bigfile > /dev/scullv0; head -5 /proc/scullvmem
Device 0: qset 500, order 4, sz 1535135
item at cfab4800, qset at cf8e4000
0:d087a000
1:d08d2000
The values show two different behaviors. On x86_64, physical addresses and virtual
addresses are mapped to completely different address ranges (0x100 and 0xffffff00),
while on x86 computers, vmalloc returns virtual addresses just above the mapping
used for physical memory.
Per-CPU Variables
Per-CPU variables are an interesting 2.6 kernel feature. When you create a per-CPU
variable, each processor on the system gets its own copy of that variable. This may
seem like a strange thing to want to do, but it has its advantages. Access to per-CPU
variables requires (almost) no locking, because each processor works with its own
copy. Per-CPU variables can also remain in their respective processors'caches, which
leads to significantly better performance for frequently updated quantities.
A good example of per-CPU variable use can be found in the networking subsystem.
The kernel maintains no end of counters tracking how many of each type of packet
was received; these counters can be updated thousands of times per second. Rather
than deal with the caching and locking issues, the networking developers put the statistics
counters into per-CPU variables. Updates are now lockless and fast. On the
rare occasion that user space requests to see the values of the counters, it is a simple
matter to add up each processor's version and return the total.
The declarations for per-CPU variables can be found in <linux/percpu.h>. To create a
per-CPU variable at compile time, use this macro:
DEFINE_PER_CPU(type, name);
If the variable (to be called name) is an array, include the dimension information with
the type. Thus, a per-CPU array of three integers would be created with:
DEFINE_PER_CPU(int[3], my_percpu_array);
Per-CPU variables can be manipulated without explicit locking--almost. Remember
that the 2.6 kernel is preemptible; it would not do for a processor to be preempted in
PAGE 229
the middle of a critical section that modifies a per-CPU variable. It also would not be
good if your process were to be moved to another processor in the middle of a perCPU
variable access. For this reason, you must explicitly use the get_cpu_var macro
to access the current processor's copy of a given variable, and call put_cpu_var when
you are done. The call to get_cpu_var returns an lvalue for the current processor's
version of the variable and disables preemption. Since an lvalue is returned, it can be
assigned to or operated on directly. For example, one counter in the networking code
is incremented with these two statements:
get_cpu_var(sockets_in_use)++;
put_cpu_var(sockets_in_use);
You can access another processor's copy of the variable with:
per_cpu(variable, int cpu_id);
If you write code that involves processors reaching into each other's per-CPU variables,
you, of course, have to implement a locking scheme that makes that access
safe.
Dynamically allocated per-CPU variables are also possible. These variables can be
allocated with:
void *alloc_percpu(type);
void *__alloc_percpu(size_t size, size_t align);
In most cases, alloc_percpu does the job; you can call __alloc_percpu in cases where
a particular alignment is required. In either case, a per-CPU variable can be returned
to the system with free_percpu. Access to a dynamically allocated per-CPU variable is
done via per_cpu_ptr:
per_cpu_ptr(void *per_cpu_var, int cpu_id);
This macro returns a pointer to the version of per_cpu_var corresponding to the given
cpu_id. If you are simply reading another CPU's version of the variable, you can dereference
that pointer and be done with it. If, however, you are manipulating the current
processor's version, you probably need to ensure that you cannot be moved out of
that processor first. If the entirety of your access to the per-CPU variable happens
with a spinlock held, all is well. Usually, however, you need to use get_cpu to block
preemption while working with the variable. Thus, code using dynamic per-CPU variables
tends to look like this:
int cpu;
cpu = get_cpu( )
ptr = per_cpu_ptr(per_cpu_var, cpu);
/* work with ptr */
put_cpu( );
When using compile-time per-CPU variables, the get_cpu_var and put_cpu_var macros
take care of these details. Dynamic per-CPU variables require more explicit protection.
PAGE 230
Per-CPU variables can be exported to modules, but you must use a special version of
the macros:
EXPORT_PER_CPU_SYMBOL(per_cpu_var);
EXPORT_PER_CPU_SYMBOL_GPL(per_cpu_var);
To access such a variable within a module, declare it with:
DECLARE_PER_CPU(type, name);
The use of DECLARE_PER_CPU (instead of DEFINE_PER_CPU) tells the compiler
that an external reference is being made.
If you want to use per-CPU variables to create a simple integer counter, take a look
at the canned implementation in <linux/percpu_counter.h>. Finally, note that some
architectures have a limited amount of address space available for per-CPU variables.
If you create per-CPU variables in your code, you should try to keep them
small.
Obtaining Large Buffers
As we have noted in previous sections, allocations of large, contiguous memory buffers
are prone to failure. System memory fragments over time, and chances are that a
truly large region of memory will simply not be available. Since there are usually
ways of getting the job done without huge buffers, the kernel developers have not
put a high priority on making large allocations work. Before you try to obtain a large
memory area, you should really consider the alternatives. By far the best way of performing
large I/O operations is through scatter/gather operations, which we discuss
in the section "Scatter-gather mappings" in Chapter 1.
Acquiring a Dedicated Buffer at Boot Time
If you really need a huge buffer of physically contiguous memory, the best approach
is often to allocate it by requesting memory at boot time. Allocation at boot time is
the only way to retrieve consecutive memory pages while bypassing the limits
imposed by __get_free_pages on the buffer size, both in terms of maximum allowed
size and limited choice of sizes. Allocating memory at boot time is a "dirty" technique,
because it bypasses all memory management policies by reserving a private
memory pool. This technique is inelegant and inflexible, but it is also the least prone
to failure. Needless to say, a module can't allocate memory at boot time; only drivers
directly linked to the kernel can do that.
One noticeable problem with boot-time allocation is that it is not a feasible option
for the average user, since this mechanism is available only for code linked in the kernel
image. A device driver using this kind of allocation can be installed or replaced
only by rebuilding the kernel and rebooting the computer.
PAGE 231
When the kernel is booted, it gains access to all the physical memory available in the
system. It then initializes each of its subsystems by calling that subsystem's initialization
function, allowing initialization code to allocate a memory buffer for private use
by reducing the amount of RAM left for normal system operation.
Boot-time memory allocation is performed by calling one of these functions:
#include <linux/bootmem.h>
void *alloc_bootmem(unsigned long size);
void *alloc_bootmem_low(unsigned long size);
void *alloc_bootmem_pages(unsigned long size);
void *alloc_bootmem_low_pages(unsigned long size);
The functions allocate either whole pages (if they end with _pages) or non-page aligned
memory areas. The allocated memory may be high memory unless one of the
_low versions is used. If you are allocating this buffer for a device driver, you probably
want to use it for DMA operations, and that is not always possible with high
memory; thus, you probably want to use one of the _low variants.
It is rare to free memory allocated at boot time; you will almost certainly be unable
to get it back later if you want it. There is an interface to free this memory, however:
void free_bootmem(unsigned long addr, unsigned long size);
Note that partial pages freed in this manner are not returned to the system--but, if
you are using this technique, you have probably allocated a fair number of whole
pages to begin with.
If you must use boot-time allocation, you need to link your driver directly into the
kernel. See the files in the kernel source under Documentation/kbuild for more information
on how this should be done.
Quick Reference
The functions and symbols related to memory allocation are:
#include <linux/slab.h>
void *kmalloc(size_t size, int flags);
void kfree(void *obj);
The most frequently used interface to memory allocation.
#include <linux/mm.h>
GFP_USER
GFP_KERNEL
GFP_NOFS
GFP_NOIO
GFP_ATOMIC
Flags that control how memory allocations are performed, from the least restrictive
to the most. The GFP_USER and GFP_KERNEL priorities allow the current process
PAGE 232
to be put to sleep to satisfy the request. GFP_NOFS and GFP_NOIO disable filesystem
operations and all I/O operations, respectively, while GFP_ATOMIC allocations cannot
sleep at all.
__GFP_DMA
__GFP_HIGHMEM
__GFP_COLD
__GFP_NOWARN
__GFP_HIGH
__GFP_REPEAT
__GFP_NOFAIL
__GFP_NORETRY
These flags modify the kernel's behavior when allocating memory.
#include <linux/malloc.h>
kmem_cache_t *kmem_cache_create(char *name, size_t size, size_t offset,
unsigned long flags, constructor( ), destructor( ));
int kmem_cache_destroy(kmem_cache_t *cache);
Create and destroy a slab cache. The cache can be used to allocate several
objects of the same size.
SLAB_NO_REAP
SLAB_HWCACHE_ALIGN
SLAB_CACHE_DMA
Flags that can be specified while creating a cache.
SLAB_CTOR_ATOMIC
SLAB_CTOR_CONSTRUCTOR
Flags that the allocator can pass to the constructor and the destructor functions.
void *kmem_cache_alloc(kmem_cache_t *cache, int flags);
void kmem_cache_free(kmem_cache_t *cache, const void *obj);
Allocate and release a single object from the cache.
/proc/slabinfo
A virtual file containing statistics on slab cache usage.
#include <linux/mempool.h>
mempool_t *mempool_create(int min_nr, mempool_alloc_t *alloc_fn, mempool_free_t *free_fn, void *data);
void mempool_destroy(mempool_t *pool);
Functions for the creation of memory pools, which try to avoid memory allocation
failures by keeping an "emergency list" of allocated items.
void *mempool_alloc(mempool_t *pool, int gfp_mask);
void mempool_free(void *element, mempool_t *pool);
Functions for allocating items from (and returning them to) memory pools.
PAGE 233
unsigned long get_zeroed_page(int flags);
unsigned long __get_free_page(int flags);
unsigned long __get_free_pages(int flags, unsigned long order);
The page-oriented allocation functions. get_zeroed_page returns a single, zero-filled
page. All the other versions of the call do not initialize the contents of the
returned page(s).
int get_order(unsigned long size);
Returns the allocation order associated to size in the current platform, according
to PAGE_SIZE. The argument must be a power of two, and the return value is at
least 0.
void free_page(unsigned long addr);
void free_pages(unsigned long addr, unsigned long order);
Functions that release page-oriented allocations.
struct page *alloc_pages_node(int nid, unsigned int flags, unsigned int order);
struct page *alloc_pages(unsigned int flags, unsigned int order);
struct page *alloc_page(unsigned int flags);
All variants of the lowest-level page allocator in the Linux kernel.
void __free_page(struct page *page);
void __free_pages(struct page *page, unsigned int order);
void free_hot_page(struct page *page);
void free_cold_page(struct page *page);
Various ways of freeing pages allocated with one of the forms of alloc_page.
#include <linux/vmalloc.h>
void * vmalloc(unsigned long size);
void vfree(void * addr);
#include <asm/io.h>
void * ioremap(unsigned long offset, unsigned long size);
void iounmap(void *addr);
Functions that allocate or free a contiguous virtual address space. ioremap
accesses physical memory through virtual addresses, while vmalloc allocates free
pages. Regions mapped with ioremap are freed with iounmap, while pages
obtained from vmalloc are released with vfree.
#include <linux/percpu.h>
DEFINE_PER_CPU(type, name);
DECLARE_PER_CPU(type, name);
Macros that define and declare per-CPU variables.
per_cpu(variable, int cpu_id)
get_cpu_var(variable)
put_cpu_var(variable)
Macros that provide access to statically declared per-CPU variables.
PAGE 234
void *alloc_percpu(type);
void *__alloc_percpu(size_t size, size_t align);
void free_percpu(void *variable);
Functions that perform runtime allocation and freeing of per-CPU variables.
int get_cpu( );
void put_cpu( );
per_cpu_ptr(void *variable, int cpu_id)
get_cpu obtains a reference to the current processor (therefore, preventing preemption
and movement to another processor) and returns the ID number of the
processor; put_cpu returns that reference. To access a dynamically allocated perCPU
variable, use per_cpu_ptr with the ID of the CPU whose version should be
accessed. Manipulations of the current CPU's version of a dynamic, per-CPU
variable should probably be surrounded by calls to get_cpu and put_cpu.
#include <linux/bootmem.h>
void *alloc_bootmem(unsigned long size);
void *alloc_bootmem_low(unsigned long size);
void *alloc_bootmem_pages(unsigned long size);
void *alloc_bootmem_low_pages(unsigned long size);
void free_bootmem(unsigned long addr, unsigned long size);
Functions (which can be used only by drivers directly linked into the kernel) that
perform allocation and freeing of memory at system bootstrap time.
PAGE 235
CHAPTER 9
Communicating with Hardware
Although playing with scull and similar toys is a good introduction to the software
interface of a Linux device driver, implementing a real device requires hardware. The
driver is the abstraction layer between software concepts and hardware circuitry; as
such, it needs to talk with both of them. Up until now, we have examined the internals
of software concepts; this chapter completes the picture by showing you how
a driver can access I/O ports and I/O memory while being portable across Linux
platforms.
This chapter continues in the tradition of staying as independent of specific hardware
as possible. However, where specific examples are needed, we use simple digital
I/O ports (such as the standard PC parallel port) to show how the I/O
instructions work and normal frame-buffer video memory to show memory-mapped
I/O.
We chose simple digital I/O because it is the easiest form of an input/output port.
Also, the parallel port implements raw I/O and is available in most computers: data
bits written to the device appear on the output pins, and voltage levels on the input
pins are directly accessible by the processor. In practice, you have to connect LEDs
or a printer to the port to actually see the results of a digital I/O operation, but the
underlying hardware is extremely easy to use.
I/O Ports and I/O Memory
Every peripheral device is controlled by writing and reading its registers. Most of the
time a device has several registers, and they are accessed at consecutive addresses,
either in the memory address space or in the I/O address space.
At the hardware level, there is no conceptual difference between memory regions and
I/O regions: both of them are accessed by asserting electrical signals on the address
PAGE 236
bus and control bus (i.e., the read and write signals)* and by reading from or writing
to the data bus.
While some CPU manufacturers implement a single address space in their chips, others
decided that peripheral devices are different from memory and, therefore, deserve
a separate address space. Some processors (most notably the x86 family) have separate
read and write electrical lines for I/O ports and special CPU instructions to
access ports.
Because peripheral devices are built to fit a peripheral bus, and the most popular I/O
buses are modeled on the personal computer, even processors that do not have a separate
address space for I/O ports must fake reading and writing I/O ports when
accessing some peripheral devices, usually by means of external chipsets or extra circuitry
in the CPU core. The latter solution is common within tiny processors meant
for embedded use.
For the same reason, Linux implements the concept of I/O ports on all computer
platforms it runs on, even on platforms where the CPU implements a single address
space. The implementation of port access sometimes depends on the specific make
and model of the host computer (because different models use different chipsets to
map bus transactions into memory address space).
Even if the peripheral bus has a separate address space for I/O ports, not all devices
map their registers to I/O ports. While use of I/O ports is common for ISA peripheral
boards, most PCI devices map registers into a memory address region. This I/O
memory approach is generally preferred, because it doesn't require the use of special purpose
processor instructions; CPU cores access memory much more efficiently,
and the compiler has much more freedom in register allocation and addressing-mode
selection when accessing memory.
I/O Registers and Conventional Memory
Despite the strong similarity between hardware registers and memory, a programmer
accessing I/O registers must be careful to avoid being tricked by CPU (or compiler)
optimizations that can modify the expected I/O behavior.
The main difference between I/O registers and RAM is that I/O operations have side
effects, while memory operations have none: the only effect of a memory write is
storing a value to a location, and a memory read returns the last value written there.
Because memory access speed is so critical to CPU performance, the no-side-effects
case has been optimized in several ways: values are cached and read/write instructions
are reordered.
* Not all computer platforms use a read and a write signal; some have different means to address external circuits. The difference is irrelevant at software level, however, and we'll assume all have read and write to simplify
the discussion.
PAGE 237
The compiler can cache data values into CPU registers without writing them to
memory, and even if it stores them, both write and read operations can operate on
cache memory without ever reaching physical RAM. Reordering can also happen
both at the compiler level and at the hardware level: often a sequence of instructions
can be executed more quickly if it is run in an order different from that which
appears in the program text, for example, to prevent interlocks in the RISC pipeline.
On CISC processors, operations that take a significant amount of time can be executed
concurrently with other, quicker ones.
These optimizations are transparent and benign when applied to conventional memory
(at least on uniprocessor systems), but they can be fatal to correct I/O operations,
because they interfere with those "side effects" that are the main reason why a
driver accesses I/O registers. The processor cannot anticipate a situation in which
some other process (running on a separate processor, or something happening inside
an I/O controller) depends on the order of memory access. The compiler or the CPU
may just try to outsmart you and reorder the operations you request; the result can
be strange errors that are very difficult to debug. Therefore, a driver must ensure that
no caching is performed and no read or write reordering takes place when accessing
registers.
The problem with hardware caching is the easiest to face: the underlying hardware is
already configured (either automatically or by Linux initialization code) to disable
any hardware cache when accessing I/O regions (whether they are memory or port
regions).
The solution to compiler optimization and hardware reordering is to place a memory
barrier between operations that must be visible to the hardware (or to another processor)
in a particular order. Linux provides four macros to cover all possible ordering
needs:
#include <linux/kernel.h>
void barrier(void)
This function tells the compiler to insert a memory barrier but has no effect on
the hardware. Compiled code stores to memory all values that are currently
modified and resident in CPU registers, and rereads them later when they are
needed. A call to barrier prevents compiler optimizations across the barrier but
leaves the hardware free to do its own reordering.
#include <asm/system.h>
void rmb(void);
void read_barrier_depends(void);
void wmb(void);
void mb(void);
These functions insert hardware memory barriers in the compiled instruction
flow; their actual instantiation is platform dependent. An rmb (read memory
barrier) guarantees that any reads appearing before the barrier are completed
PAGE 238
prior to the execution of any subsequent read. wmb guarantees ordering in write
operations, and the mb instruction guarantees both. Each of these functions is a
superset of barrier.
read_barrier_depends is a special, weaker form of read barrier. Whereas rmb prevents
the reordering of all reads across the barrier, read_barrier_depends blocks
only the reordering of reads that depend on data from other reads. The distinction
is subtle, and it does not exist on all architectures. Unless you understand
exactly what is going on, and you have a reason to believe that a full read barrier
is exacting an excessive performance cost, you should probably stick to using
rmb.
void smp_rmb(void);
void smp_read_barrier_depends(void);
void smp_wmb(void);
void smp_mb(void);
These versions of the barrier macros insert hardware barriers only when the kernel
is compiled for SMP systems; otherwise, they all expand to a simple barrier
call.
A typical usage of memory barriers in a device driver may have this sort of form:
writel(dev->registers.addr, io_destination_address);
writel(dev->registers.size, io_size);
writel(dev->registers.operation, DEV_READ);
wmb( );
writel(dev->registers.control, DEV_GO);
In this case, it is important to be sure that all of the device registers controlling a particular
operation have been properly set prior to telling it to begin. The memory barrier
enforces the completion of the writes in the necessary order.
Because memory barriers affect performance, they should be used only where they
are really needed. The different types of barriers can also have different performance
characteristics, so it is worthwhile to use the most specific type possible. For example,
on the x86 architecture, wmb( ) currently does nothing, since writes outside the
processor are not reordered. Reads are reordered, however, so mb( ) is slower than
wmb( ).
It is worth noting that most of the other kernel primitives dealing with synchronization,
such as spinlock and atomic_t operations, also function as memory barriers.
Also worthy of note is that some peripheral buses (such as the PCI bus) have caching
issues of their own; we discuss those when we get to them in later chapters.
Some architectures allow the efficient combination of an assignment and a memory
barrier. The kernel provides a few macros that perform this combination; in the
default case, they are defined as follows:
#define set_mb(var, value) do {var = value; mb( );} while 0
#define set_wmb(var, value) do {var = value; wmb( );} while 0
#define set_rmb(var, value) do {var = value; rmb( );} while 0
PAGE 239
Where appropriate, <asm/system.h> defines these macros to use architecture-specific
instructions that accomplish the task more quickly. Note that set_rmb is defined
only by a small number of architectures. (The use of a do...while construct is a standard
C idiom that causes the expanded macro to work as a normal C statement in all
contexts.)
Using I/O Ports
I/O ports are the means by which drivers communicate with many devices, at least
part of the time. This section covers the various functions available for making use of
I/O ports; we also touch on some portability issues.
I/O Port Allocation
As you might expect, you should not go off and start pounding on I/O ports without
first ensuring that you have exclusive access to those ports. The kernel provides a
registration interface that allows your driver to claim the ports it needs. The core
function in that interface is request_region:
#include <linux/ioport.h>
struct resource *request_region(unsigned long first, unsigned long n,
const char *name);
This function tells the kernel that you would like to make use of n ports, starting
with first. The name parameter should be the name of your device. The return value
is non-NULL if the allocation succeeds. If you get NULL back from request_region, you
will not be able to use the desired ports.
All port allocations show up in /proc/ioports. If you are unable to allocate a needed
set of ports, that is the place to look to see who got there first.
When you are done with a set of I/O ports (at module unload time, perhaps), they
should be returned to the system with:
void release_region(unsigned long start, unsigned long n);
There is also a function that allows your driver to check to see whether a given set of
I/O ports is available:
int check_region(unsigned long first, unsigned long n);
Here, the return value is a negative error code if the given ports are not available.
This function is deprecated because its return value provides no guarantee of
whether an allocation would succeed; checking and later allocating are not an atomic
operation. We list it here because several drivers are still using it, but you should
always use request_region, which performs the required locking to ensure that the
allocation is done in a safe, atomic manner.
PAGE 240
Manipulating I/O ports
After a driver has requested the range of I/O ports it needs to use in its activities, it
must read and/or write to those ports. To this end, most hardware differentiates
between 8-bit, 16-bit, and 32-bit ports. Usually you can't mix them like you normally
do with system memory access.*
A C program, therefore, must call different functions to access different size ports. As
suggested in the previous section, computer architectures that support only memory-mapped
I/O registers fake port I/O by remapping port addresses to memory
addresses, and the kernel hides the details from the driver in order to ease portability.
The Linux kernel headers (specifically, the architecture-dependent header <asm/
io.h>) define the following inline functions to access I/O ports:
unsigned inb(unsigned port);
void outb(unsigned char byte, unsigned port);
Read or write byte ports (eight bits wide). The port argument is defined as
unsigned long for some platforms and unsigned short for others. The return
type of inb is also different across architectures.
unsigned inw(unsigned port);
void outw(unsigned short word, unsigned port);
These functions access 16-bit ports (one word wide); they are not available when
compiling for the S390 platform, which supports only byte I/O.
unsigned inl(unsigned port);
void outl(unsigned longword, unsigned port);
These functions access 32-bit ports. longword is declared as either unsigned long
or unsigned int, according to the platform. Like word I/O, "long" I/O is not
available on S390.
From now on, when we use unsigned without further type specifications,
we are referring to an architecture-dependent definition whose
exact nature is not relevant. The functions are almost always portable,
because the compiler automatically casts the values during assignment--their
being unsigned helps prevent compile-time warnings. No
information is lost with such casts as long as the programmer assigns
sensible values to avoid overflow. We stick to this convention of
"incomplete typing" throughout this chapter.
Note that no 64-bit port I/O operations are defined. Even on 64-bit architectures, the
port address space uses a 32-bit (maximum) data path.
* Sometimes I/O ports are arranged like memory, and you can (for example) bind two 8-bit writes into a single
16-bit operation. This applies, for instance, to PC video boards. But generally, you can't count on this feature.
PAGE 241
I/O Port Access from User Space
The functions just described are primarily meant to be used by device drivers, but
they can also be used from user space, at least on PC-class computers. The GNU C
library defines them in <sys/io.h>. The following conditions should apply in order
for inb and friends to be used in user-space code:
- The program must be compiled with the -O option to force expansion of inline functions.
- The ioperm or iopl system calls must be used to get permission to perform I/O
operations on ports. ioperm gets permission for individual ports, while iopl gets
permission for the entire I/O space. Both of these functions are x86-specific.
- The program must run as root to invoke ioperm or iopl.* Alternatively, one of its
ancestors must have gained port access running as root.
If the host platform has no ioperm and no iopl system calls, user space can still access
I/O ports by using the /dev/port device file. Note, however, that the meaning of the
file is very platform-specific and not likely useful for anything but the PC.
The sample sources misc-progs/inp.c and misc-progs/outp.c are a minimal tool for
reading and writing ports from the command line, in user space. They expect to be
installed under multiple names (e.g., inb, inw, and inl and manipulates byte, word, or
long ports depending on which name was invoked by the user). They use ioperm or
iopl under x86, /dev/port on other platforms.
The programs can be made setuid root, if you want to live dangerously and play with
your hardware without acquiring explicit privileges. Please do not install them setuid
on a production system, however; they are a security hole by design.
String Operations
In addition to the single-shot in and out operations, some processors implement special
instructions to transfer a sequence of bytes, words, or longs to and from a single
I/O port or the same size. These are the so-called string instructions, and they perform
the task more quickly than a C-language loop can do. The following macros
implement the concept of string I/O either by using a single machine instruction or
by executing a tight loop if the target processor has no instruction that performs
string I/O. The macros are not defined at all when compiling for the S390 platform.
This should not be a portability problem, since this platform doesn't usually share
device drivers with other platforms, because its peripheral buses are different.
* Technically, it must have the CAP_SYS_RAWIO capability, but that is the same as running as root on most current systems.
PAGE 242
The prototypes for string functions are:
void insb(unsigned port, void *addr, unsigned long count);
void outsb(unsigned port, void *addr, unsigned long count);
Read or write count bytes starting at the memory address addr. Data is read from
or written to the single port port.
void insw(unsigned port, void *addr, unsigned long count);
void outsw(unsigned port, void *addr, unsigned long count);
Read or write 16-bit values to a single 16-bit port.
void insl(unsigned port, void *addr, unsigned long count);
void outsl(unsigned port, void *addr, unsigned long count);
Read or write 32-bit values to a single 32-bit port.
There is one thing to keep in mind when using the string functions: they move a
straight byte stream to or from the port. When the port and the host system have different
byte ordering rules, the results can be surprising. Reading a port with inw
swaps the bytes, if need be, to make the value read match the host ordering. The
string functions, instead, do not perform this swapping.
Pausing I/O
Some platforms--most notably the i386--can have problems when the processor
tries to transfer data too quickly to or from the bus. The problems can arise when the
processor is overclocked with respect to the peripheral bus (think ISA here) and can
show up when the device board is too slow. The solution is to insert a small delay
after each I/O instruction if another such instruction follows. On the x86, the pause
is achieved by performing an out b instruction to port 0x80 (normally but not always
unused), or by busy waiting. See the io.h file under your platform's asm subdirectory
for details.
If your device misses some data, or if you fear it might miss some, you can use pausing
functions in place of the normal ones. The pausing functions are exactly like
those listed previously, but their names end in _p; they are called inb_p, outb_p, and
so on. The functions are defined for most supported architectures, although they
often expand to the same code as nonpausing I/O, because there is no need for the
extra pause if the architecture runs with a reasonably modern peripheral bus.
Platform Dependencies
I/O instructions are, by their nature, highly processor dependent. Because they work
with the details of how the processor handles moving data in and out, it is very hard
to hide the differences between systems. As a consequence, much of the source code
related to port I/O is platform-dependent.
You can see one of the incompatibilities, data typing, by looking back at the list of functions,
where the arguments are typed differently based on the architectural differences
PAGE 243
between platforms. For example, a port is unsigned short on the x86 (where the processor
supports a 64-KB I/O space), but unsigned long on other platforms, whose ports are
just special locations in the same address space as memory.
Other platform dependencies arise from basic structural differences in the processors
and are, therefore, unavoidable. We won't go into detail about the differences,
because we assume that you won't be writing a device driver for a particular system
without understanding the underlying hardware. Instead, here is an overview of the
capabilities of the architectures supported by the kernel:
IA-32 (x86)
x86_64
The architecture supports all the functions described in this chapter. Port numbers
are of type unsigned short.
IA-64 (Itanium)
All functions are supported; ports are unsigned long (and memory-mapped).
String functions are implemented in C.
Alpha
All the functions are supported, and ports are memory-mapped. The implementation
of port I/O is different in different Alpha platforms, according to the
chipset they use. String functions are implemented in C and defined in arch/
alpha/lib/io.c. Ports are unsigned long.
ARM
Ports are memory-mapped, and all functions are supported; string functions are
implemented in C. Ports are of type unsigned int.
Cris
This architecture does not support the I/O port abstraction even in an emulated
mode; the various port operations are defined to do nothing at all.
M68k
M68k-nommu
Ports are memory-mapped. String functions are supported, and the port type is
unsigned char *.
MIPS
MIPS64
The MIPS port supports all the functions. String operations are implemented
with tight assembly loops, because the processor lacks machine-level string I/O.
Ports are memory-mapped; they are unsigned long.
PA-RISC
All of the functions are supported; ports are int on PCI-based systems and
unsigned short on EISA systems, except for string operations,
which use unsigned long port numbers.
PAGE 244
PowerPC
PowerPC64
All the functions are supported; ports have type unsigned char * on 32-bit systems
and unsigned long on 64-bit systems.
S390
Similar to the M68k, the header for this platform supports only byte-wide port I/O
with no string operations. Ports are char pointers and are memory-mapped.
Super-H
Ports are unsigned int (memory-mapped), and all the functions are supported.
SPARC
SPARC64
Once again, I/O space is memory-mapped. Versions of the port functions are
defined to work with unsigned long ports.
The curious reader can extract more information from the io.h files, which sometimes
define a few architecture-specific functions in addition to those we describe in
this chapter. Be warned that some of these files are rather difficult reading, however.
It's interesting to note that no processor outside the x86 family features a different
address space for ports, even though several of the supported families are shipped
with ISA and/or PCI slots (and both buses implement separate I/O and memory
address spaces).
Moreover, some processors (most notably the early Alphas) lack instructions that
move one or two bytes at a time.* Therefore, their peripheral chipsets simulate 8-bit
and 16-bit I/O accesses by mapping them to special address ranges in the memory
address space. Thus, an inb and an inw instruction that act on the same port are
implemented by two 32-bit memory reads that operate on different addresses. Fortunately,
all of this is hidden from the device driver writer by the internals of the macros
described in this section, but we feel it's an interesting feature to note. If you
want to probe further, look for examples in include/asm-alpha/core_lca.h.
How I/O operations are performed on each platform is well described in the programmer's
manual for each platform; those manuals are usually available for download
as PDFs on the Web.
* Single-byte I/O is not as important as one may imagine, because it is a rare operation. To read/write a single
byte to any address space, you need to implement a data path connecting the low bits of the register-set data
bus to any byte position in the external data bus. These data paths require additional logic gates that get in
the way of every data transfer. Dropping byte-wide loads and stores can benefit overall system performance.
PAGE 245
An I/O Port Example
The sample code we use to show port I/O from within a device driver acts on general-purpose
digital I/O ports; such ports are found in most computer systems.
A digital I/O port, in its most common incarnation, is a byte-wide I/O location,
either memory-mapped or port-mapped. When you write a value to an output location,
the electrical signal seen on output pins is changed according to the individual
bits being written. When you read a value from the input location, the current logic
level seen on input pins is returned as individual bit values.
The actual implementation and software interface of such I/O ports varies from system
to system. Most of the time, I/O pins are controlled by two I/O locations: one
that allows selecting what pins are used as input and what pins are used as output
and one in which you can actually read or write logic levels. Sometimes, however,
things are even simpler, and the bits are hardwired as either input or output (but, in
this case, they're no longer called "general-purpose I/O"); the parallel port found on
all personal computers is one such not-so-general-purpose I/O port. Either way, the
I/O pins are usable by the sample code we introduce shortly.
An Overview of the Parallel Port
Because we expect most readers to be using an x86 platform in the form called "personal
computer," we feel it is worth explaining how the PC parallel port is designed.
The parallel port is the peripheral interface of choice for running digital I/O sample
code on a personal computer. Although most readers probably have parallel port
specifications available, we summarize them here for your convenience.
The parallel interface, in its minimal configuration (we overlook the ECP and EPP
modes) is made up of three 8-bit ports. The PC standard starts the I/O ports for the
first parallel interface at 0x378 and for the second at 0x278. The first port is a bidirectional
data register; it connects directly to pins 2-9 on the physical connector. The
second port is a read-only status register; when the parallel port is being used for a
printer, this register reports several aspects of printer status, such as being online,
out of paper, or busy. The third port is an output-only control register, which,
among other things, controls whether interrupts are enabled.
The signal levels used in parallel communications are standard transistor-transistor
logic (TTL) levels: 0 and 5 volts, with the logic threshold at about 1.2 volts. You can
count on the ports at least meeting the standard TTL LS current ratings, although
most modern parallel ports do better in both current and voltage ratings.
PAGE 246
The parallel connector is not isolated from the computer's internal circuitry,
which is useful if you want to connect logic gates directly to the
port. But you have to be careful to do the wiring correctly; the parallel
port circuitry is easily damaged when you play with your own custom
circuitry, unless you add optoisolators to your circuit. You can choose
to use plug-in parallel ports if you fear you'll damage your motherboard.
The bit specifications are outlined in Figure 9-1. You can access 12 output bits and 5
input bits, some of which are logically inverted over the course of their signal path.
The only bit with no associated signal pin is bit 4 (0x10) of port 2, which enables
interrupts from the parallel port. We use this bit as part of our implementation of an
interrupt handler in Chapter 10.
Figure 9-1. The pinout of the parallel port
A Sample Driver
The driver we introduce is called short (Simple Hardware Operations and Raw
Tests). All it does is read and write a few 8-bit ports, starting from the one you select
at load time. By default, it uses the port range assigned to the parallel interface of the
PC. Each device node (with a unique minor number) accesses a different port. The
short driver doesn't do anything useful; it just isolates for external use as a single
instruction acting on a port. If you are not used to port I/O, you can use short to get
PAGE 247
familiar with it; you can measure the time it takes to transfer data through a port or
play other games.
For short to work on your system, it must have free access to the underlying hardware
device (by default, the parallel interface); thus, no other driver may have allocated
it. Most modern distributions set up the parallel port drivers as modules that
are loaded only when needed, so contention for the I/O addresses is not usually a
problem. If, however, you get a "can't get I/O address" error from short (on the console
or in the system log file), some other driver has probably already taken the port.
A quicklook at /proc/ioports usually tells you which driver is getting in the way. The
same caveat applies to other I/O devices if you are not using the parallel interface.
From now on, we just refer to "the parallel interface" to simplify the discussion.
However, you can set the base module parameter at load time to redirect short to
other I/O devices. This feature allows the sample code to run on any Linux platform
where you have access to a digital I/O interface that is accessible via outb and inb
(even though the actual hardware is memory-mapped on all platforms but the x86).
Later, in the section "Using I/O Memory," we show how short can be used with
generic memory-mapped digital I/O as well.
To watch what happens on the parallel connector and if you have a bit of an inclination
to work with hardware, you can solder a few LEDs to the output pins. Each
LED should be connected in series to a 1-K resistor leading to a ground pin (unless,
of course, your LEDs have the resistor built in). If you connect an output pin to an
input pin, you'll generate your own input to be read from the input ports.
Note that you cannot just connect a printer to the parallel port and see data sent to
short. This driver implements simple access to the I/O ports and does not perform
the handshake that printers need to operate on the data. In the next chapter, we
show a sample driver (called shortprint), that is capable of driving parallel printers;
that driver uses interrupts, however, so we can't get to it quite yet.
If you are going to view parallel data by soldering LEDs to a D-type connector, we
suggest that you not use pins 9 and 10, because we connect them together later to
run the sample code shown in Chapter 10.
As far as short is concerned, /dev/short0 writes to and reads from the 8-bit port
located at the I/O address base (0x378 unless changed at load time). /dev/short1
writes to the 8-bit port located at base + 1, and so on up to base + 7.
The actual output operation performed by /dev/short0 is based on a tight loop using
outb. A memory barrier instruction is used to ensure that the output operation actually
takes place and is not optimized away:
while (count--) {
outb(*(ptr++), port);
wmb( );
}
PAGE 248
You can run the following command to light your LEDs:
echo -n "any string" > /dev/short0
Each LED monitors a single bit of the output port. Remember that only the last character
written remains steady on the output pins long enough to be perceived by your
eyes. For that reason, we suggest that you prevent automatic insertion of a trailing
newline by passing the -n option to echo.
Reading is performed by a similar function, built around inb instead of outb. In order
to read "meaningful" values from the parallel port, you need to have some hardware
connected to the input pins of the connector to generate signals. If there is no signal,
you read an endless stream of identical bytes. If you choose to read from an output
port, you most likely get back the last value written to the port (this applies to the
parallel interface and to most other digital I/O circuits in common use). Thus, those
uninclined to get out their soldering irons can read the current output value on port
0x378 by running a command such as:
dd if=/dev/short0 bs=1 count=1 | od -t x1
To demonstrate the use of all the I/O instructions, there are three variations of each
short device: /dev/short0 performs the loop just shown, /dev/short0p uses outb_p and
inb_p in place of the "fast" functions, and /dev/short0s uses the string instructions.
There are eight such devices, from short0 to short7. Although the PC parallel interface
has only three ports, you may need more of them if using a different I/O device
to run your tests.
The short driver performs an absolute minimum of hardware control but is adequate
to show how the I/O port instructions are used. Interested readers may want to look
at the source for the parport and parport_pc modules to see how complicated this
device can get in real life in order to support a range of devices (printers, tape
backup, network interfaces) on the parallel port.
Using I/O Memory
Despite the popularity of I/O ports in the x86 world, the main mechanism used to
communicate with devices is through memory-mapped registers and device memory.
Both are called I/O memory because the difference between registers and memory
is transparent to software.
I/O memory is simply a region of RAM-like locations that the device makes available
to the processor over the bus. This memory can be used for a number of purposes,
such as holding video data or Ethernet packets, as well as implementing device registers
that behave just like I/O ports (i.e., they have side effects associated with reading
and writing them).
The way to access I/O memory depends on the computer architecture, bus, and
device being used, although the principles are the same everywhere. The discussion
PAGE 249
in this chapter touches mainly on ISA and PCI memory, while trying to convey general
information as well. Although access to PCI memory is introduced here, a thorough
discussion of PCI is deferred to Chapter 12.
Depending on the computer platform and bus being used, I/O memory may or may
not be accessed through page tables. When access passes though page tables, the
kernel must first arrange for the physical address to be visible from your driver, and
this usually means that you must call ioremap before doing any I/O. If no page tables
are needed, I/O memory locations look pretty much like I/O ports, and you can just
read and write to them using proper wrapper functions.
Whether or not ioremap is required to access I/O memory, direct use of pointers to I/O
memory is discouraged. Even though (as introduced in the section "I/O Ports and I/O
Memory") I/O memory is addressed like normal RAM at hardware level, the extra
care outlined in the section "I/O Registers and Conventional Memory" suggests
avoiding normal pointers. The wrapper functions used to access I/O memory are safe
on all platforms and are optimized away whenever straight pointer dereferencing can
perform the operation.
Therefore, even though dereferencing a pointer works (for now) on the x86, failure
to use the proper macros hinders the portability and readability of the driver.
I/O Memory Allocation and Mapping
I/O memory regions must be allocated prior to use. The interface for allocation of
memory regions (defined in <linux/ioport.h>) is:
struct resource *request_mem_region(unsigned long start, unsigned long len,
char *name);
This function allocates a memory region of len bytes, starting at start. If all goes
well, a non-NULL pointer is returned; otherwise the return value is NULL. All I/O memory
allocations are listed in /proc/iomem.
Memory regions should be freed when no longer needed:
void release_mem_region(unsigned long start, unsigned long len);
There is also an old function for checking I/O memory region availability:
int check_mem_region(unsigned long start, unsigned long len);
But, as with check_region, this function is unsafe and should be avoided.
Allocation of I/O memory is not the only required step before that memory may be
accessed. You must also ensure that this I/O memory has been made accessible to the
kernel. Getting at I/O memory is not just a matter of dereferencing a pointer; on many
systems, I/O memory is not directly accessible in this way at all. So a mapping must
be set up first. This is the role of the ioremap function, introduced in the section
PAGE 250
"vmalloc and Friends" in Chapter 1. The function is designed specifically to assign
virtual addresses to I/O memory regions.
Once equipped with ioremap (and iounmap), a device driver can access any I/O
memory address, whether or not it is directly mapped to virtual address space.
Remember, though, that the addresses returned from ioremap should not be dereferenced
directly; instead, accessor functions provided by the kernel should be used.
Before we get into those functions, we'd better review the ioremap prototypes and
introduce a few details that we passed over in the previous chapter.
The functions are called according to the following definition:
#include <asm/io.h>
void *ioremap(unsigned long phys_addr, unsigned long size);
void *ioremap_nocache(unsigned long phys_addr, unsigned long size);
void iounmap(void * addr);
First of all, you notice the new function ioremap_nocache. We didn't cover it in
Chapter 8, because its meaning is definitely hardware related. Quoting from one of
the kernel headers: "It's useful if some control registers are in such an area, and write
combining or read caching is not desirable." Actually, the function's implementation
is identical to ioremap on most computer platforms: in situations where all of I/O
memory is already visible through noncacheable addresses, there's no reason to
implement a separate, noncaching version of ioremap.
Accessing I/O Memory
On some platforms, you may get away with using the return value from ioremap as a
pointer. Such use is not portable, and, increasingly, the kernel developers have been
working to eliminate any such use. The proper way of getting at I/O memory is via a
set of functions (defined via <asm/io.h>) provided for that purpose.
To read from I/O memory, use one of the following:
unsigned int ioread8(void *addr);
unsigned int ioread16(void *addr);
unsigned int ioread32(void *addr);
Here, addr should be an address obtained from ioremap (perhaps with an integer offset);
the return value is what was read from the given I/O memory.
There is a similar set of functions for writing to I/O memory:
void iowrite8(u8 value, void *addr);
void iowrite16(u16 value, void *addr);
void iowrite32(u32 value, void *addr);
If you must read or write a series of values to a given I/O memory address, you can
use the repeating versions of the functions:
void ioread8_rep(void *addr, void *buf, unsigned long count);
void ioread16_rep(void *addr, void *buf, unsigned long count);
PAGE 251
void ioread32_rep(void *addr, void *buf, unsigned long count);
void iowrite8_rep(void *addr, const void *buf, unsigned long count);
void iowrite16_rep(void *addr, const void *buf, unsigned long count);
void iowrite32_rep(void *addr, const void *buf, unsigned long count);
These functions read or write count values from the given buf to the given addr. Note
that count is expressed in the size of the data being written; ioread32_rep reads count
32-bit values starting at buf.
The functions described above perform all I/O to the given addr. If, instead, you need
to operate on a block of I/O memory, you can use one of the following:
void memset_io(void *addr, u8 value, unsigned int count);
void memcpy_fromio(void *dest, void *source, unsigned int count);
void memcpy_toio(void *dest, void *source, unsigned int count);
These functions behave like their C library analogs.
If you read through the kernel source, you see many calls to an older set of functions
when I/O memory is being used. These functions still work, but their use in new
code is discouraged. Among other things, they are less safe because they do not perform
the same sort of type checking. Nonetheless, we describe them here:
unsigned readb(address);
unsigned readw(address);
unsigned readl(address);
These macros are used to retrieve 8-bit, 16-bit, and 32-bit data values from I/O
memory.
void writeb(unsigned value, address);
void writew(unsigned value, address);
void writel(unsigned value, address);
Like the previous functions, these functions (macros) are used to write 8-bit, 16bit,
and 32-bit data items.
Some 64-bit platforms also offer readq and writeq, for quad-word (8-byte) memory
operations on the PCI bus. The quad-word nomenclature is a historical leftover from
the times when all real processors had 16-bit words. Actually, the L naming used for
32-bit values has become incorrect too, but renaming everything would confuse
things even more.
Ports as I/O Memory
Some hardware has an interesting feature: some versions use I/O ports, while others
use I/O memory. The registers exported to the processor are the same in either case,
but the access method is different. As a way of making life easier for drivers dealing
with this kind of hardware, and as a way of minimizing the apparent differences
between I/O port and memory accesses, the 2.6 kernel provides a function called
ioport_map:
void *ioport_map(unsigned long port, unsigned int count);
PAGE 252
This function remaps count I/O ports and makes them appear to be I/O memory.
From that point thereafter, the driver may use ioread8 and friends on the returned
addresses and forget that it is using I/O ports at all.
This mapping should be undone when it is no longer needed:
void ioport_unmap(void *addr);
These functions make I/O ports look like memory. Do note, however, that the I/O
ports must still be allocated with request_region before they can be remapped in this
way.
Reusing short for I/O Memory
The short sample module, introduced earlier to access I/O ports, can be used to
access I/O memory as well. To this aim, you must tell it to use I/O memory at load
time; also, you need to change the base address to make it point to your I/O region.
For example, this is how we used short to light the debug LEDs on a MIPS development
board:
mips.root# ./short_load use_mem=1 base=0xb7ffffc0
mips.root# echo -n 7 > /dev/short0
Use of short for I/O memory is the same as it is for I/O ports.
The following fragment shows the loop used by short in writing to a memory location:
while (count--) {
iowrite8(*ptr++, address);
wmb( );
}
Note the use of a write memory barrier here. Because iowrite8 likely turns into a
direct assignment on many architectures, the memory barrier is needed to ensure
that the writes happen in the expected order.
short uses inb and outb to show how that is done. It would be a straightforward exercise
for the reader, however, to change short to remap I/O ports with ioport_map,
and simplify the rest of the code considerably.
ISA Memory Below 1 MB
One of the most well-known I/O memory regions is the ISA range found on personal
computers. This is the memory range between 640 KB (0xA0000) and 1 MB
(0x100000). Therefore, it appears right in the middle of regular system RAM. This
positioning may seem a little strange; it is an artifact of a decision made in the early
1980s, when 640 KB of memory seemed like more than anybody would ever be able
to use.
PAGE 253
This memory range belongs to the non-directly-mapped class of memory.* You can
read/write a few bytes in that memory range using the short module as explained previously,
that is, by setting use_mem at load time.
Although ISA I/O memory exists only in x86-class computers, we thinkit's worth
spending a few words and a sample driver on it.
We are not going to discuss PCI memory in this chapter, since it is the cleanest kind
of I/O memory: once you know the physical address, you can simply remap and
access it. The "problem" with PCI I/O memory is that it doesn't lend itself to a working
example for this chapter, because we can't know in advance the physical
addresses your PCI memory is mapped to, or whether it's safe to access either of
those ranges. We chose to describe the ISA memory range, because it's both less
clean and more suitable to running sample code.
To demonstrate access to ISA memory, we use yet another silly little module (part of
the sample sources). In fact, this one is called silly, as an acronym for Simple Tool for
Unloading and Printing ISA Data, or something like that.
The module supplements the functionality of short by giving access to the whole
384-KB memory space and by showing all the different I/O functions. It features four
device nodes that perform the same task using different data transfer functions. The
silly devices act as a window over I/O memory, in a way similar to /dev/mem. You
can read and write data, and lseek to an arbitrary I/O memory address.
Because silly provides access to ISA memory, it must start by mapping the physical
ISA addresses into kernel virtual addresses. In the early days of the Linux kernel, one
could simply assign a pointer to an ISA address of interest, then dereference it
directly. In the modern world, though, we must work with the virtual memory system
and remap the memory range first. This mapping is done with ioremap, as
explained earlier for short:
#define ISA_BASE 0xA0000
#define ISA_MAX 0x100000 /* for general memory access */
/* this line appears in silly_init */
io_base = ioremap(ISA_BASE, ISA_MAX - ISA_BASE);
ioremap returns a pointer value that can be used with ioread8 and the other functions
explained in the section "Accessing I/O Memory."
Let's look back at our sample module to see how these functions might be used. /dev/
sillyb, featuring minor number 0, accesses I/O memory with ioread8 and iowrite8.
The following code shows the implementation for read, which makes the address
* Actually, this is not completely true. The memory range is so small and so frequently used that the kernel
builds page tables at boot time to access those addresses. However, the virtual address used to access them
is not the same as the physical address, and thus ioremap is needed anyway.
PAGE 254
range 0xA0000-0xFFFFF available as a virtual file in the range 0-0x5FFFF. The read
function is structured as a switch statement over the different access modes; here is
the sillyb case:
case M_8:
while (count) {
*ptr = ioread8(add);
add++;
count--;
ptr++;
}
break;
The next two devices are /dev/sillyw (minor number 1) and /dev/sillyl (minor number
2). They act like /dev/sillyb, except that they use 16-bit and 32-bit functions. Here's
the write implementation of sillyl, again part of a switch:
case M_32:
while (count >= 4) {
iowrite8(*(u32 *)ptr, add);
add += 4;
count -= 4;
ptr += 4;
}
break;
The last device is /dev/sillycp (minor number 3), which uses the memcpy_*io functions
to perform the same task. Here's the core of its read implementation:
case M_memcpy:
memcpy_fromio(ptr, add, count);
break;
Because ioremap was used to provide access to the ISA memory area, silly must
invoke iounmap when the module is unloaded:
iounmap(io_base);
isa_readb and Friends
A look at the kernel source will turn up another set of routines with names such as
isa_readb. In fact, each of the functions just described has an isa_ equivalent. These
functions provide access to ISA memory without the need for a separate ioremap
step. The word from the kernel developers, however, is that these functions are
intended to be temporary driver-porting aids and that they may go away in the
future. Therefore, you should avoid using them.
PAGE 255
Quick Reference
This chapter introduced the following symbols related to hardware management:
#include <linux/kernel.h>
void barrier(void)
This "software" memory barrier requests the compiler to consider all memory
volatile across this instruction.
#include <asm/system.h>
void rmb(void);
void read_barrier_depends(void);
void wmb(void);
void mb(void);
Hardware memory barriers. They request the CPU (and the compiler) to checkpoint
all memory reads, writes, or both across this instruction.
#include <asm/io.h>
unsigned inb(unsigned port);
void outb(unsigned char byte, unsigned port);
unsigned inw(unsigned port);
void outw(unsigned short word, unsigned port);
unsigned inl(unsigned port);
void outl(unsigned doubleword, unsigned port);
Functions that are used to read and write I/O ports. They can also be called by
user-space programs, provided they have the right privileges to access ports.
unsigned inb_p(unsigned port);
...
If a small delay is needed after an I/O operation, you can use the six pausing
counterparts of the functions introduced in the previous entry; these pausing
functions have names ending in _p.
void insb(unsigned port, void *addr, unsigned long count);
void outsb(unsigned port, void *addr, unsigned long count);
void insw(unsigned port, void *addr, unsigned long count);
void outsw(unsigned port, void *addr, unsigned long count);
void insl(unsigned port, void *addr, unsigned long count);
void outsl(unsigned port, void *addr, unsigned long count);
The "string functions" are optimized to transfer data from an input port to a
region of memory, or the other way around. Such transfers are performed by
reading or writing the same port count times.
PAGE 256
#include <linux/ioport.h>
struct resource *request_region(unsigned long start, unsigned long len, char *name);
void release_region(unsigned long start, unsigned long len);
int check_region(unsigned long start, unsigned long len);
Resource allocators for I/O ports. The (deprecated) check function returns 0 for
success and less than 0 in case of error.
struct resource *request_mem_region(unsigned long start, unsigned long len, char *name);
void release_mem_region(unsigned long start, unsigned long len);
int check_mem_region(unsigned long start, unsigned long len);
Functions that handle resource allocation for memory regions.
#include <asm/io.h>
void *ioremap(unsigned long phys_addr, unsigned long size);
void *ioremap_nocache(unsigned long phys_addr, unsigned long size);
void iounmap(void *virt_addr);
ioremap remaps a physical address range into the processor's virtual address
space, making it available to the kernel. iounmap frees the mapping when it is no
longer needed.
#include <asm/io.h>
unsigned int ioread8(void *addr);
unsigned int ioread16(void *addr);
unsigned int ioread32(void *addr);
void iowrite8(u8 value, void *addr);
void iowrite16(u16 value, void *addr);
void iowrite32(u32 value, void *addr);
Accessor functions that are used to work with I/O memory.
void ioread8_rep(void *addr, void *buf, unsigned long count);
void ioread16_rep(void *addr, void *buf, unsigned long count);
void ioread32_rep(void *addr, void *buf, unsigned long count);
void iowrite8_rep(void *addr, const void *buf, unsigned long count);
void iowrite16_rep(void *addr, const void *buf, unsigned long count);
void iowrite32_rep(void *addr, const void *buf, unsigned long count);
"Repeating" versions of the I/O memory primitives.
PAGE 257
unsigned readb(address);
unsigned readw(address);
unsigned readl(address);
void writeb(unsigned value, address);
void writew(unsigned value, address);
void writel(unsigned value, address);
memset_io(address, value, count);
memcpy_fromio(dest, source, nbytes);
memcpy_toio(dest, source, nbytes);
Older, type-unsafe functions for accessing I/O memory.
void *ioport_map(unsigned long port, unsigned int count);
void ioport_unmap(void *addr);
A driver author that wants to treat I/O ports as if they were I/O memory may pass
those ports to ioport_map. The mapping should be done (with ioport_unmap)
when no longer needed.
PAGE 258
CHAPTER 10
Interrupt Handling
Although some devices can be controlled using nothing but their I/O regions, most
real devices are a bit more complicated than that. Devices have to deal with the
external world, which often includes things such as spinning disks, moving tape,
wires to distant places, and so on. Much has to be done in a time frame that is different
from, and far slower than, that of the processor. Since it is almost always undesirable
to have the processor wait on external events, there must be a way for a device
to let the processor know when something has happened.
That way, of course, is interrupts. An interrupt is simply a signal that the hardware
can send when it wants the processor's attention. Linux handles interrupts in much
the same way that it handles signals in user space. For the most part, a driver need
only register a handler for its device's interrupts, and handle them properly when
they arrive. Of course, underneath that simple picture there is some complexity; in
particular, interrupt handlers are somewhat limited in the actions they can perform
as a result of how they are run.
It is difficult to demonstrate the use of interrupts without a real hardware device to
generate them. Thus, the sample code used in this chapter works with the parallel
port. Such ports are starting to become scarce on modern hardware, but, with luck,
most people are still able to get their hands on a system with an available port. We'll
be working with the short module from the previous chapter; with some small additions
it can generate and handle interrupts from the parallel port. The module's
name, short, actually means short int (it is C, isn't it?), to remind us that it handles
interrupts.
Before we get into the topic, however, it is time for one cautionary note. Interrupt
handlers, by their nature, run concurrently with other code. Thus, they inevitably
raise issues of concurrency and contention for data structures and hardware. If you
succumbed to the temptation to pass over the discussion in Chapter 5, we understand.
But we also recommend that you turn back and have another look now. A
solid understanding of concurrency control techniques is vital when working with
interrupts.
PAGE 259
Preparing the Parallel Port
Although the parallel interface is simple, it can trigger interrupts. This capability is
used by the printer to notify the lp driver that it is ready to accept the next character
in the buffer.
Like most devices, the parallel port doesn't actually generate interrupts before it's
instructed to do so; the parallel standard states that setting bit 4 of port 2 (0x37a,
0x27a, or whatever) enables interrupt reporting. A simple outb call to set the bit is
performed by short at module initialization.
Once interrupts are enabled, the parallel interface generates an interrupt whenever
the electrical signal at pin 10 (the so-called ACK bit) changes from low to high. The
simplest way to force the interface to generate interrupts (short of hooking up a
printer to the port) is to connect pins 9 and 10 of the parallel connector. A short
length of wire inserted into the appropriate holes in the parallel port connector on
the back of your system creates this connection. The pinout of the parallel port is
shown in Figure 9-1.
Pin 9 is the most significant bit of the parallel data byte. If you write binary data to
/dev/short0, you generate several interrupts. Writing ASCII text to the port won't
generate any interrupts, though, because the ASCII character set has no entries with
the top bit set.
If you'd rather avoid wiring pins together, but you do have a printer at hand, you can
run the sample interrupt handler using a real printer, as shown later. However, note
that the probing functions we introduce depend on the jumper between pin 9 and 10
being in place, and you need it to experiment with probing using our code.
Installing an Interrupt Handler
If you want to actually "see" interrupts being generated, writing to the hardware
device isn't enough; a software handler must be configured in the system. If the
Linux kernel hasn't been told to expect your interrupt, it simply acknowledges and
ignores it.
Interrupt lines are a precious and often limited resource, particularly when there are
only 15 or 16 of them. The kernel keeps a registry of interrupt lines, similar to the
registry of I/O ports. A module is expected to request an interrupt channel (or IRQ,
for interrupt request) before using it and to release it when finished. In many situations,
modules are also expected to be able to share interrupt lines with other drivers,
as we will see. The following functions, declared in <linux/interrupt.h>,
implement the interrupt registration interface:
int request_irq(unsigned int irq,
irqreturn_t (*handler)(int, void *, struct pt_regs *),
unsigned long flags,
PAGE 260
const char *dev_name,
void *dev_id);
void free_irq(unsigned int irq, void *dev_id);
The value returned from request_irq to the requesting function is either 0 to indicate
success or a negative error code, as usual. It's not uncommon for the function to
return -EBUSY to signal that another driver is already using the requested interrupt
line. The arguments to the functions are as follows:
unsigned int irq
The interrupt number being requested.
irqreturn_t (*handler)(int, void *, struct pt_regs *)
The pointer to the handling function being installed. We discuss the arguments
to this function and its return value later in this chapter.
unsigned long flags
As you might expect, a bit maskof options (described later) related to interrupt
management.
const char *dev_name
The string passed to request_irq is used in /proc/interrupts to show the owner of
the interrupt (see the next section).
void *dev_id
Pointer used for shared interrupt lines. It is a unique identifier that is used when
the interrupt line is freed and that may also be used by the driver to point to its
own private data area (to identify which device is interrupting). If the interrupt is
not shared, dev_id can be set to NULL, but it a good idea anyway to use this item
to point to the device structure. We'll see a practical use for dev_id in the section
"Implementing a Handler."
The bits that can be set in flags are as follows:
SA_INTERRUPT
When set, this indicates a "fast" interrupt handler. Fast handlers are executed
with interrupts disabled on the current processor (the topic is covered in the section
"Fast and Slow Handlers").
SA_SHIRQ
This bit signals that the interrupt can be shared between devices. The concept of
sharing is outlined in the section "Interrupt Sharing."
SA_SAMPLE_RANDOM
This bit indicates that the generated interrupts can contribute to the entropy pool
used by /dev/random and /dev/urandom. These devices return truly random numbers
when read and are designed to help application software choose secure keys
for encryption. Such random numbers are extracted from an entropy pool that is
contributed by various random events. If your device generates interrupts at truly
random times, you should set this flag. If, on the other hand, your interrupts are
PAGE 261
predictable (for example, vertical blanking of a frame grabber), the flag is not
worth setting--it wouldn't contribute to system entropy anyway. Devices that
could be influenced by attackers should not set this flag; for example, network
drivers can be subjected to predictable packet timing from outside and should not
contribute to the entropy pool. See the comments in drivers/char/random.c for
more information.
The interrupt handler can be installed either at driver initialization or when the
device is first opened. Although installing the interrupt handler from within the module's
initialization function might sound like a good idea, it often isn't, especially if
your device does not share interrupts. Because the number of interrupt lines is limited,
you don't want to waste them. You can easily end up with more devices in your
computer than there are interrupts. If a module requests an IRQ at initialization, it
prevents any other driver from using the interrupt, even if the device holding it is
never used. Requesting the interrupt at device open, on the other hand, allows some
sharing of resources.
It is possible, for example, to run a frame grabber on the same interrupt as a modem,
as long as you don't use the two devices at the same time. It is quite common for
users to load the module for a special device at system boot, even if the device is
rarely used. A data acquisition gadget might use the same interrupt as the second
serial port. While it's not too hard to avoid connecting to your Internet service provider
(ISP) during data acquisition, being forced to unload a module in order to use
the modem is really unpleasant.
The correct place to call request_irq is when the device is first opened, before the
hardware is instructed to generate interrupts. The place to call free_irq is the last
time the device is closed, after the hardware is told not to interrupt the processor any
more. The disadvantage of this technique is that you need to keep a per-device open
count so that you know when interrupts can be disabled.
This discussion notwithstanding, short requests its interrupt line at load time. This
was done so that you can run the test programs without having to run an extra process
to keep the device open. short, therefore, requests the interrupt from within its
initialization function (short_init) instead of doing it in short_open, as a real device
driver would.
The interrupt requested by the following code is short_irq. The actual assignment of
the variable (i.e., determining which IRQ to use) is shown later, since it is not relevant
to the current discussion. short_base is the base I/O address of the parallel interface
being used; register 2 of the interface is written to enable interrupt reporting.
if (short_irq >= 0) {
result = request_irq(short_irq, short_interrupt,
SA_INTERRUPT, "short", NULL);
if (result) {
printk(KERN_INFO "short: can't get assigned irq %i\n",
short_irq);
PAGE 262
short_irq = -1;
}
else { /* actually enable it -- assume this *is* a parallel port */
outb(0x10,short_base+2);
}
}
The code shows that the handler being installed is a fast handler (SA_INTERRUPT),
doesn't support interrupt sharing (SA_SHIRQ is missing), and doesn't contribute to
system entropy (SA_SAMPLE_RANDOM is missing, too). The outb call then enables interrupt
reporting for the parallel port.
For what it's worth, the i386 and x86_64 architectures define a function for querying
the availability of an interrupt line:
int can_request_irq(unsigned int irq, unsigned long flags);
This function returns a nonzero value if an attempt to allocate the given interrupt succeeds.
Note, however, that things can always change between calls to can_request_irq
and request_irq.
The /proc Interface
Whenever a hardware interrupt reaches the processor, an internal counter is incremented,
providing a way to check whether the device is working as expected.
Reported interrupts are shown in /proc/interrupts. The following snapshot was taken
on a two-processor Pentium system:
root@montalcino:/bike/corbet/write/ldd3/src/short# m /proc/interrupts
CPU0 CPU1
0: 4848108 34 IO-APIC-edge timer
2: 0 0 XT-PIC cascade
8: 3 1 IO-APIC-edge rtc
10: 4335 1 IO-APIC-level aic7xxx
11: 8903 0 IO-APIC-level uhci_hcd
12: 49 1 IO-APIC-edge i8042
NMI: 0 0
LOC: 4848187 4848186
ERR: 0
MIS: 0
The first column is the IRQ number. You can see from the IRQs that are missing that
the file shows only interrupts corresponding to installed handlers. For example, the
first serial port (which uses interrupt number 4) is not shown, indicating that the
modem isn't being used. In fact, even if the modem had been used earlier but wasn't
in use at the time of the snapshot, it would not show up in the file; the serial ports
are well behaved and release their interrupt handlers when the device is closed.
The /proc/interrupts display shows how many interrupts have been delivered to each
CPU on the system. As you can see from the output, the Linux kernel generally handles
PAGE 263
interrupts on the first CPU as a way of maximizing cache locality.* The last two columns
give information on the programmable interrupt controller that handles the interrupt
(and that a driver writer does not need to worry about), and the name(s) of the
device(s) that have registered handlers for the interrupt (as specified in the dev_name
argument to request_irq).
The /proc tree contains another interrupt-related file, /proc/stat; sometimes you'll
find one file more useful and sometimes you'll prefer the other. /proc/stat records
several low-level statistics about system activity, including (but not limited to) the
number of interrupts received since system boot. Each line of stat begins with a text
string that is the key to the line; the intr markis what we are looking for. The following
(truncated) snapshot was taken shortly after the previous one:
intr 5167833 5154006 2 0 2 4907 0 2 68 4 0 4406 9291 50 0 0
The first number is the total of all interrupts, while each of the others represents a
single IRQ line, starting with interrupt 0. All of the counts are summed across all
processors in the system. This snapshot shows that interrupt number 4 has been
used 4907 times, even though no handler is currently installed. If the driver you're
testing acquires and releases the interrupt at each open and close cycle, you may find
/proc/stat more useful than /proc/interrupts.
Another difference between the two files is that interrupts is not architecture dependent
(except, perhaps, for a couple of lines at the end), whereas stat is; the number of
fields depends on the hardware underlying the kernel. The number of available interrupts
varies from as few as 15 on the SPARC to as many as 256 on the IA-64 and a
few other systems. It's interesting to note that the number of interrupts defined on
the x86 is currently 224, not 16 as you may expect; this, as explained in include/
asm-i386/irq.h, depends on Linux using the architectural limit instead of an implementation-specific
limit (such as the 16 interrupt sources of the old-fashioned PC
interrupt controller).
The following is a snapshot of /proc/interrupts taken on an IA-64 system. As you can
see, besides different hardware routing of common interrupt sources, the output is
very similar to that from the 32-bit system shown earlier.
CPU0 CPU1
27: 1705 34141 IO-SAPIC-level qla1280
40: 0 0 SAPIC perfmon
43: 913 6960 IO-SAPIC-level eth0
47: 26722 146 IO-SAPIC-level usb-uhci
64: 3 6 IO-SAPIC-edge ide0
80: 4 2 IO-SAPIC-edge keyboard
89: 0 0 IO-SAPIC-edge PS/2 Mouse
239: 5606341 5606052 SAPIC timer
* Although, some larger systems explicitly use interrupt balancing schemes to spread the
interrupt load across the system.
PAGE 264
254: 67575 52815 SAPIC IPI
NMI: 0 0
ERR: 0
Auto-Detecting the IRQ Number
One of the most challenging problems for a driver at initialization time can be how
to determine which IRQ line is going to be used by the device. The driver needs the
information in order to correctly install the handler. Even though a programmer
could require the user to specify the interrupt number at load time, this is a bad practice,
because most of the time the user doesn't know the number, either because he
didn't configure the jumpers or because the device is jumperless. Most users want
their hardware to "just work" and are not interested in issues like interrupt numbers.
So auto-detection of the interrupt number is a basic requirement for driver
usability.
Sometimes auto-detection depends on the knowledge that some devices feature a
default behavior that rarely, if ever, changes. In this case, the driver might assume
that the default values apply. This is exactly how short behaves by default with the
parallel port. The implementation is straightforward, as shown by short itself:
if (short_irq < 0) /* not yet specified: force the default on */
switch(short_base) {
case 0x378: short_irq = 7; break;
case 0x278: short_irq = 2; break;
case 0x3bc: short_irq = 5; break;
}
The code assigns the interrupt number according to the chosen base I/O address,
while allowing the user to override the default at load time with something like:
insmod ./short.ko irq=x
short_base defaults to 0x378, so short_irq defaults to 7.
Some devices are more advanced in design and simply "announce" which interrupt
they're going to use. In this case, the driver retrieves the interrupt number by reading
a status byte from one of the device's I/O ports or PCI configuration space.
When the target device is one that has the ability to tell the driver which interrupt it
is going to use, auto-detecting the IRQ number just means probing the device, with
no additional work required to probe the interrupt. Most modern hardware works
this way, fortunately; for example, the PCI standard solves the problem by requiring
peripheral devices to declare what interrupt line(s) they are going to use. The PCI
standard is discussed in Chapter 12.
Unfortunately, not every device is programmer friendly, and auto-detection might
require some probing. The technique is quite simple: the driver tells the device to
generate interrupts and watches what happens. If everything goes well, only one
interrupt line is activated.
PAGE 265
Although probing is simple in theory, the actual implementation might be unclear.
We look at two ways to perform the task: calling kernel-defined helper functions and
implementing our own version.
Kernel-assisted probing
The Linux kernel offers a low-level facility for probing the interrupt number. It works
for only nonshared interrupts, but most hardware that is capable of working in a
shared interrupt mode provides better ways of finding the configured interrupt number
anyway. The facility consists of two functions, declared in <linux/interrupt.h>
(which also describes the probing machinery):
unsigned long probe_irq_on(void);
This function returns a bit maskof unassigned interrupts. The driver must preserve
the returned bit mask, and pass it to probe_irq_off later. After this call, the
driver should arrange for its device to generate at least one interrupt.
int probe_irq_off(unsigned long);
After the device has requested an interrupt, the driver calls this function, passing
as its argument the bit maskpreviously returned by probe_irq_on. probe_irq_off
returns the number of the interrupt that was issued after "probe_on." If no interrupts
occurred, 0 is returned (therefore, IRQ 0 can't be probed for, but no custom
device can use it on any of the supported architectures anyway). If more than
one interrupt occurred (ambiguous detection), probe_irq_off returns a negative
value.
The programmer should be careful to enable interrupts on the device after the call to
probe_irq_on and to disable them before calling probe_irq_off. Additionally, you
must remember to service the pending interrupt in your device after probe_irq_off.
The short module demonstrates how to use such probing. If you load the module
with probe=1, the following code is executed to detect your interrupt line, provided
pins 9 and 10 of the parallel connector are bound together:
int count = 0;
do {
unsigned long mask;
mask = probe_irq_on( );
outb_p(0x10,short_base+2); /* enable reporting */
outb_p(0x00,short_base); /* clear the bit */
outb_p(0xFF,short_base); /* set the bit: interrupt! */
outb_p(0x00,short_base+2); /* disable reporting */
udelay(5); /* give it some time */
short_irq = probe_irq_off(mask);
if (short_irq = = 0) { /* none of them? */
printk(KERN_INFO "short: no irq reported by probe\n");
short_irq = -1;
}
PAGE 266
/*
* if more than one line has been activated, the result is
* negative. We should service the interrupt (no need for lpt port)
* and loop over again. Loop at most five times, then give up
*/
} while (short_irq < 0 && count++ < 5);
if (short_irq < 0)
printk("short: probe failed %i times, giving up\n", count);
Note the use of udelay before calling probe_irq_off. Depending on the speed of your
processor, you may have to wait for a brief period to give the interrupt time to actually
be delivered.
Probing might be a lengthy task. While this is not true for short, probing a frame
grabber, for example, requires a delay of at least 20 ms (which is ages for the processor),
and other devices might take even longer. Therefore, it's best to probe for the
interrupt line only once, at module initialization, independently of whether you
install the handler at device open (as you should) or within the initialization function
(which is not recommended).
It's interesting to note that on some platforms (PowerPC, M68k, most MIPS implementations,
and both SPARC versions) probing is unnecessary, and, therefore, the
previous functions are just empty placeholders, sometimes called "useless ISA nonsense."
On other platforms, probing is implemented only for ISA devices. Anyway,
most architectures define the functions (even if they are empty) to ease porting existing
device drivers.
Do-it-yourself probing
Probing can also be implemented in the driver itself without too much trouble. It is a
rare driver that must implement its own probing, but seeing how it works gives some
insight into the process. To that end, the short module performs do-it-yourself detection
of the IRQ line if it is loaded with probe=2.
The mechanism is the same as the one described earlier: enable all unused interrupts,
then wait and see what happens. We can, however, exploit our knowledge of
the device. Often a device can be configured to use one IRQ number from a set of
three or four; probing just those IRQs enables us to detect the right one, without
having to test for all possible IRQs.
The short implementation assumes that 3, 5, 7, and 9 are the only possible IRQ values.
These numbers are actually the values that some parallel devices allow you to
select.
The following code probes by testing all "possible" interrupts and looking at what
happens. The trials array lists the IRQs to try and has 0 as the end marker; the
tried array is used to keep track of which handlers have actually been registered by
this driver.
PAGE 267
int trials[ ] = {3, 5, 7, 9, 0};
int tried[ ] = {0, 0, 0, 0, 0};
int i, count = 0;
/*
* install the probing handler for all possible lines. Remember
* the result (0 for success, or -EBUSY) in order to only free
* what has been acquired
*/
for (i = 0; trials[i]; i++)
tried[i] = request_irq(trials[i], short_probing,
SA_INTERRUPT, "short probe", NULL);
do {
short_irq = 0; /* none got, yet */
outb_p(0x10,short_base+2); /* enable */
outb_p(0x00,short_base);
outb_p(0xFF,short_base); /* toggle the bit */
outb_p(0x00,short_base+2); /* disable */
udelay(5); /* give it some time */
/* the value has been set by the handler */
if (short_irq = = 0) { /* none of them? */
printk(KERN_INFO "short: no irq reported by probe\n");
}
/*
* If more than one line has been activated, the result is
* negative. We should service the interrupt (but the lpt port
* doesn't need it) and loop over again. Do it at most 5 times
*/
} while (short_irq <=0 && count++ < 5);
/* end of loop, uninstall the handler */
for (i = 0; trials[i]; i++)
if (tried[i] = = 0)
free_irq(trials[i], NULL);
if (short_irq < 0)
printk("short: probe failed %i times, giving up\n", count);
You might not know in advance what the "possible" IRQ values are. In that case,
you need to probe all the free interrupts, instead of limiting yourself to a few trials[ ].
To probe for all interrupts, you have to probe from IRQ 0 to IRQ NR_IRQS-1, where
NR_IRQS is defined in <asm/irq.h> and is platform dependent.
Now we are missing only the probing handler itself. The handler's role is to update
short_irq according to which interrupts are actually received. A 0 value in short_irq
means "nothing yet," while a negative value means "ambiguous." These values were
chosen to be consistent with probe_irq_off and to allow the same code to call either
kind of probing within short.c.
irqreturn_t short_probing(int irq, void *dev_id, struct pt_regs *regs)
{
PAGE 268
if (short_irq = = 0) short_irq = irq; /* found */
if (short_irq != irq) short_irq = -irq; /* ambiguous */
return IRQ_HANDLED;
}
The arguments to the handler are described later. Knowing that irq is the interrupt
being handled should be sufficient to understand the function just shown.
Fast and Slow Handlers
Older versions of the Linux kernel took great pains to distinguish between "fast" and
"slow" interrupts. Fast interrupts were those that could be handled very quickly,
whereas handling slow interrupts took significantly longer. Slow interrupts could be
sufficiently demanding of the processor, and it was worthwhile to reenable interrupts
while they were being handled. Otherwise, tasks requiring quick attention
could be delayed for too long.
In modern kernels, most of the differences between fast and slow interrupts have disappeared.
There remains only one: fast interrupts (those that were requested with
the SA_INTERRUPT flag) are executed with all other interrupts disabled on the current
processor. Note that other processors can still handle interrupts, although you will
never see two processors handling the same IRQ at the same time.
So, which type of interrupt should your driver use? On modern systems, SA_INTERRUPT is
intended only for use in a few, specific situations such as timer interrupts. Unless you
have a strong reason to run your interrupt handler with other interrupts disabled, you
should not use SA_INTERRUPT.
This description should satisfy most readers, although someone with a taste for hardware
and some experience with her computer might be interested in going deeper. If
you don't care about the internal details, you can skip to the next section.
The internals of interrupt handling on the x86
This description has been extrapolated from arch/i386/kernel/irq.c, arch/i386/kernel/
apic.c, arch/i386/kernel/entry.S, arch/i386/kernel/i8259.c, and include/asm-i386/hw_irq.h
as they appear in the 2.6 kernels; although the general concepts remain the same, the
hardware details differ on other platforms.
The lowest level of interrupt handling can be found in entry.S, an assembly-language
file that handles much of the machine-level work. By way of a bit of assembler trickery
and some macros, a bit of code is assigned to every possible interrupt. In each
case, the code pushes the interrupt number on the stack and jumps to a common
segment, which calls do_IRQ, defined in irq.c.
The first thing do_IRQ does is to acknowledge the interrupt so that the interrupt controller
can go on to other things. It then obtains a spinlock for the given IRQ number,
thus preventing any other CPU from handling this IRQ. It clears a couple of status
PAGE 269
bits (including one called IRQ_WAITING that we'll look at shortly) and then looks up the
handler(s) for this particular IRQ. If there is no handler, there's nothing to do; the
spinlock is released, any pending software interrupts are handled, and do_IRQ
returns.
Usually, however, if a device is interrupting, there is at least one handler registered
for its IRQ as well. The function handle_IRQ_event is called to actually invoke the
handlers. If the handler is of the slow variety (SA_INTERRUPT is not set), interrupts are
reenabled in the hardware, and the handler is invoked. Then it's just a matter of
cleaning up, running software interrupts, and getting back to regular work. The "regular
work" may well have changed as a result of an interrupt (the handler could
wake_up a process, for example), so the last thing that happens on return from an
interrupt is a possible rescheduling of the processor.
Probing for IRQs is done by setting the IRQ_WAITING status bit for each IRQ that currently
lacks a handler. When the interrupt happens, do_IRQ clears that bit and then
returns, because no handler is registered. probe_irq_off, when called by a driver,
needs to search for only the IRQ that no longer has IRQ_WAITING set.
Implementing a Handler
So far, we've learned to register an interrupt handler but not to write one. Actually,
there's nothing unusual about a handler--it's ordinary C code.
The only peculiarity is that a handler runs at interrupt time and, therefore, suffers
some restrictions on what it can do. These restrictions are the same as those we saw
with kernel timers. A handler can't transfer data to or from user space, because it
doesn't execute in the context of a process. Handlers also cannot do anything that
would sleep, such as calling wait_event, allocating memory with anything other than
GFP_ATOMIC, or locking a semaphore. Finally, handlers cannot call schedule.
The role of an interrupt handler is to give feedback to its device about interrupt
reception and to read or write data according to the meaning of the interrupt being
serviced. The first step usually consists of clearing a bit on the interface board; most
hardware devices won't generate other interrupts until their "interrupt-pending" bit
has been cleared. Depending on how your hardware works, this step may need to be
performed last instead of first; there is no catch-all rule here. Some devices don't
require this step, because they don't have an "interrupt-pending" bit; such devices
are a minority, although the parallel port is one of them. For that reason, short does
not have to clear such a bit.
A typical task for an interrupt handler is awakening processes sleeping on the device
if the interrupt signals the event they're waiting for, such as the arrival of new data.
To stick with the frame grabber example, a process could acquire a sequence of
images by continuously reading the device; the read call blocks before reading each
PAGE 270
frame, while the interrupt handler awakens the process as soon as each new frame
arrives. This assumes that the grabber interrupts the processor to signal successful
arrival of each new frame.
The programmer should be careful to write a routine that executes in a minimum
amount of time, independent of its being a fast or slow handler. If a long computation
needs to be performed, the best approach is to use a tasklet or workqueue to
schedule computation at a safer time (we'll look at how work can be deferred in this
manner in the section "Top and Bottom Halves.")
Our sample code in short responds to the interrupt by calling do_gettimeofday and
printing the current time into a page-sized circular buffer. It then awakens any reading
process, because there is now data available to be read.
irqreturn_t short_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
struct timeval tv;
int written;
do_gettimeofday(&tv);
/* Write a 16 byte record. Assume PAGE_SIZE is a multiple of 16 */
written = sprintf((char *)short_head,"%08u.%06u\n",
(int)(tv.tv_sec % 100000000), (int)(tv.tv_usec));
BUG_ON(written != 16);
short_incr_bp(&short_head, written);
wake_up_interruptible(&short_queue); /* awake any reading process */
return IRQ_HANDLED;
}
This code, though simple, represents the typical job of an interrupt handler. It, in
turn, calls short_incr_bp, which is defined as follows:
static inline void short_incr_bp(volatile unsigned long *index, int delta)
{
unsigned long new = *index + delta;
barrier( ); /* Don't optimize these two together */
*index = (new >= (short_buffer + PAGE_SIZE)) ? short_buffer : new;
}
This function has been carefully written to wrap a pointer into the circular buffer
without ever exposing an incorrect value. The barrier call is there to block compiler
optimizations across the other two lines of the function. Without the barrier, the
compiler might decide to optimize out the new variable and assign directly to *index.
That optimization could expose an incorrect value of the index for a brief period in
the case where it wraps. By taking care to prevent in inconsistent value from ever
being visible to other threads, we can manipulate the circular buffer pointers safely
without locks.
The device file used to read the buffer being filled at interrupt time is /dev/shortint.
This device special file, together with /dev/shortprint, wasn't introduced in
PAGE 271
Chapter 9, because its use is specific to interrupt handling. The internals of /dev/
shortint are specifically tailored for interrupt generation and reporting. Writing to the
device generates one interrupt every other byte; reading the device gives the time
when each interrupt was reported.
If you connect together pins 9 and 10 of the parallel connector, you can generate
interrupts by raising the high bit of the parallel data byte. This can be accomplished
by writing binary data to /dev/short0 or by writing anything to /dev/shortint.*
The following code implements read and write for /dev/shortint:
ssize_t short_i_read (struct file *filp, char __user *buf, size_t count,
loff_t *f_pos)
{
int count0;
DEFINE_WAIT(wait);
while (short_head = = short_tail) {
prepare_to_wait(&short_queue, &wait, TASK_INTERRUPTIBLE);
if (short_head = = short_tail)
schedule( );
finish_wait(&short_queue, &wait);
if (signal_pending (current)) /* a signal arrived */
return -ERESTARTSYS; /* tell the fs layer to handle it */
}
/* count0 is the number of readable data bytes */
count0 = short_head - short_tail;
if (count0 < 0) /* wrapped */
count0 = short_buffer + PAGE_SIZE - short_tail;
if (count0 < count) count = count0;
if (copy_to_user(buf, (char *)short_tail, count))
return -EFAULT;
short_incr_bp (&short_tail, count);
return count;
}
ssize_t short_i_write (struct file *filp, const char __user *buf, size_t count,
loff_t *f_pos)
{
int written = 0, odd = *f_pos & 1;
unsigned long port = short_base; /* output to the parallel data latch */
void *address = (void *) short_base;
if (use_mem) {
while (written < count)
iowrite8(0xff * ((++written + odd) & 1), address);
} else {
* The shortint device accomplishes its task by alternately writing 0x00 and 0xff to the parallel port.
PAGE 272
while (written < count)
outb(0xff * ((++written + odd) & 1), port);
}
*f_pos += count;
return written;
}
The other device special file, /dev/shortprint, uses the parallel port to drive a printer;
you can use it if you want to avoid connecting pins 9 and 10 of a D-25 connector.
The write implementation of shortprint uses a circular buffer to store data to be
printed, while the read implementation is the one just shown (so you can read the
time your printer takes to eat each character).
In order to support printer operation, the interrupt handler has been slightly modified
from the one just shown, adding the ability to send the next data byte to the
printer if there is more data to transfer.
Handler Arguments and Return Value
Though short ignores them, three arguments are passed to an interrupt handler: irq,
dev_id, and regs. Let's look at the role of each.
The interrupt number (int irq) is useful as information you may print in your log
messages, if any. The second argument, void *dev_id, is a sort of client data; a void *
argument is passed to request_irq, and this same pointer is then passed back as an
argument to the handler when the interrupt happens. You usually pass a pointer to
your device data structure in dev_id, so a driver that manages several instances of the
same device doesn't need any extra code in the interrupt handler to find out which
device is in charge of the current interrupt event.
Typical use of the argument in an interrupt handler is as follows:
static irqreturn_t sample_interrupt(int irq, void *dev_id, struct pt_regs
*regs)
{
struct sample_dev *dev = dev_id;
/* now `dev' points to the right hardware item */
/* .... */
}
The typical open code associated with this handler looks like this:
static void sample_open(struct inode *inode, struct file *filp)
{
struct sample_dev *dev = hwinfo + MINOR(inode->i_rdev);
request_irq(dev->irq, sample_interrupt,
0 /* flags */, "sample", dev /* dev_id */);
/*....*/
return 0;
}
PAGE 273
The last argument, struct pt_regs *regs, is rarely used. It holds a snapshot of the
processor's context before the processor entered interrupt code. The registers can be
used for monitoring and debugging; they are not normally needed for regular device
driver tasks.
Interrupt handlers should return a value indicating whether there was actually an
interrupt to handle. If the handler found that its device did, indeed, need attention, it
should return IRQ_HANDLED; otherwise the return value should be IRQ_NONE. You can
also generate the return value with this macro:
IRQ_RETVAL(handled)
where handled is nonzero if you were able to handle the interrupt. The return value is
used by the kernel to detect and suppress spurious interrupts. If your device gives
you no way to tell whether it really interrupted, you should return IRQ_HANDLED.
Enabling and Disabling Interrupts
There are times when a device driver must block the delivery of interrupts for a
(hopefully short) period of time (we saw one such situation in the section "Spinlocks"
in Chapter 5). Often, interrupts must be blocked while holding a spinlock to
avoid deadlocking the system. There are ways of disabling interrupts that do not
involve spinlocks. But before we discuss them, note that disabling interrupts should
be a relatively rare activity, even in device drivers, and this technique should never be
used as a mutual exclusion mechanism within a driver.
Disabling a single interrupt
Sometimes (but rarely!) a driver needs to disable interrupt delivery for a specific
interrupt line. The kernel offers three functions for this purpose, all declared in
<asm/irq.h>. These functions are part of the kernel API, so we describe them, but
their use is discouraged in most drivers. Among other things, you cannot disable
shared interrupt lines, and, on modern systems, shared interrupts are the norm. That
said, here they are:
void disable_irq(int irq);
void disable_irq_nosync(int irq);
void enable_irq(int irq);
Calling any of these functions may update the maskfor the specified irq in the programmable
interrupt controller (PIC), thus disabling or enabling the specified IRQ
across all processors. Calls to these functions can be nested--if disable_irq is called
twice in succession, two enable_irq calls are required before the IRQ is truly reenabled.
It is possible to call these functions from an interrupt handler, but enabling
your own IRQ while handling it is not usually good practice.
disable_irq not only disables the given interrupt but also waits for a currently executing
interrupt handler, if any, to complete. Be aware that if the thread calling disable_irq
PAGE 274
holds any resources (such as spinlocks) that the interrupt handler needs, the system
can deadlock. disable_irq_nosync differs from disable_irq in that it returns immediately.
Thus, using disable_irq_nosync is a little faster but may leave your driver open to
race conditions.
But why disable an interrupt? Sticking to the parallel port, let's look at the plip network interface.
A plip device uses the bare-bones parallel port to transfer data. Since
only five bits can be read from the parallel connector, they are interpreted as four
data bits and a clock/handshake signal. When the first four bits of a packet are transmitted
by the initiator (the interface sending the packet), the clock line is raised,
causing the receiving interface to interrupt the processor. The plip handler is then
invoked to deal with newly arrived data.
After the device has been alerted, the data transfer proceeds, using the handshake
line to clocknew data to the receiving interface (this might not be the best implementation,
but it is necessary for compatibility with other packet drivers using the
parallel port). Performance would be unbearable if the receiving interface had to handle
two interrupts for every byte received. Therefore, the driver disables the interrupt
during the reception of the packet; instead, a poll-and-delay loop is used to
bring in the data.
Similarly, because the handshake line from the receiver to the transmitter is used to
acknowledge data reception, the transmitting interface disables its IRQ line during
packet transmission.
Disabling all interrupts
What if you need to disable all interrupts? In the 2.6 kernel, it is possible to turn off
all interrupt handling on the current processor with either of the following two functions
(which are defined in <asm/system.h>):
void local_irq_save(unsigned long flags);
void local_irq_disable(void);
A call to local_irq_save disables interrupt delivery on the current processor after saving
the current interrupt state into flags. Note that flags is passed directly, not by
pointer. local_irq_disable shuts off local interrupt delivery without saving the state;
you should use this version only if you know that interrupts have not already been
disabled elsewhere.
Turning interrupts back on is accomplished with:
void local_irq_restore(unsigned long flags);
void local_irq_enable(void);
The first version restores that state which was stored into flags by local_irq_save,
while local_irq_enable enables interrupts unconditionally. Unlike disable_irq,
local_irq_disable does not keep track of multiple calls. If more than one function in
the call chain might need to disable interrupts, local_irq_save should be used.
PAGE 275
In the 2.6 kernel, there is no way to disable all interrupts globally across the entire
system. The kernel developers have decided that the cost of shutting off all interrupts
is too high and that there is no need for that capability in any case. If you are
working with an older driver that makes calls to functions such as cli and sti, you
need to update it to use proper locking before it will work under 2.6.
Top and Bottom Halves
One of the main problems with interrupt handling is how to perform lengthy tasks
within a handler. Often a substantial amount of work must be done in response to a
device interrupt, but interrupt handlers need to finish up quickly and not keep interrupts
blocked for long. These two needs (work and speed) conflict with each other,
leaving the driver writer in a bit of a bind.
Linux (along with many other systems) resolves this problem by splitting the interrupt
handler into two halves. The so-called top half is the routine that actually
responds to the interrupt--the one you register with request_irq. The bottom half is a
routine that is scheduled by the top half to be executed later, at a safer time. The big
difference between the top-half handler and the bottom half is that all interrupts are
enabled during execution of the bottom half--that's why it runs at a safer time. In
the typical scenario, the top half saves device data to a device-specific buffer, schedules
its bottom half, and exits: this operation is very fast. The bottom half then performs
whatever other work is required, such as awakening processes, starting up
another I/O operation, and so on. This setup permits the top half to service a new
interrupt while the bottom half is still working.
Almost every serious interrupt handler is split this way. For instance, when a network interface
reports the arrival of a new packet, the handler just retrieves the data
and pushes it up to the protocol layer; actual processing of the packet is performed
in a bottom half.
The Linux kernel has two different mechanisms that may be used to implement bottom-half
processing, both of which were introduced in Chapter 7. Tasklets are often
the preferred mechanism for bottom-half processing; they are very fast, but all tasklet
code must be atomic. The alternative to tasklets is workqueues, which may have a
higher latency but that are allowed to sleep.
The following discussion works, once again, with the short driver. When loaded with
a module option, short can be told to do interrupt processing in a top/bottom-half
mode with either a tasklet or workqueue handler. In this case, the top half executes
quickly; it simply remembers the current time and schedules the bottom half processing.
The bottom half is then charged with encoding this time and awakening any
user processes that may be waiting for data.
PAGE 276
Tasklets
Remember that tasklets are a special function that may be scheduled to run, in software
interrupt context, at a system-determined safe time. They may be scheduled to
run multiple times, but tasklet scheduling is not cumulative; the tasklet runs only
once, even if it is requested repeatedly before it is launched. No tasklet ever runs in
parallel with itself, since they run only once, but tasklets can run in parallel with
other tasklets on SMP systems. Thus, if your driver has multiple tasklets, they must
employ some sort of locking to avoid conflicting with each other.
Tasklets are also guaranteed to run on the same CPU as the function that first schedules
them. Therefore, an interrupt handler can be secure that a tasklet does not begin
executing before the handler has completed. However, another interrupt can certainly
be delivered while the tasklet is running, so locking between the tasklet and
the interrupt handler may still be required.
Tasklets must be declared with the DECLARE_TASKLET macro:
DECLARE_TASKLET(name, function, data);
name is the name to be given to the tasklet, function is the function that is called to
execute the tasklet (it takes one unsigned long argument and returns void), and data
is an unsigned long value to be passed to the tasklet function.
The short driver declares its tasklet as follows:
void short_do_tasklet(unsigned long);
DECLARE_TASKLET(short_tasklet, short_do_tasklet, 0);
The function tasklet_schedule is used to schedule a tasklet for running. If short is
loaded with tasklet=1, it installs a different interrupt handler that saves data and
schedules the tasklet as follows:
irqreturn_t short_tl_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
do_gettimeofday((struct timeval *) tv_head); /* cast to stop 'volatile' warning
*/
short_incr_tv(&tv_head);
tasklet_schedule(&short_tasklet);
short_wq_count++; /* record that an interrupt arrived */
return IRQ_HANDLED;
}
The actual tasklet routine, short_do_tasklet, will be executed shortly (so to speak) at
the system's convenience. As mentioned earlier, this routine performs the bulk of the
work of handling the interrupt; it looks like this:
void short_do_tasklet (unsigned long unused)
{
int savecount = short_wq_count, written;
short_wq_count = 0; /* we have already been removed from the queue */
/*
* The bottom half reads the tv array, filled by the top half,
PAGE 277
* and prints it to the circular text buffer, which is then consumed
* by reading processes
*/
/* First write the number of interrupts that occurred before this bh */
written = sprintf((char *)short_head,"bh after %6i\n",savecount);
short_incr_bp(&short_head, written);
/*
* Then, write the time values. Write exactly 16 bytes at a time,
* so it aligns with PAGE_SIZE
*/
do {
written = sprintf((char *)short_head,"%08u.%06u\n",
(int)(tv_tail->tv_sec % 100000000),
(int)(tv_tail->tv_usec));
short_incr_bp(&short_head, written);
short_incr_tv(&tv_tail);
} while (tv_tail != tv_head);
wake_up_interruptible(&short_queue); /* awake any reading process */
}
Among other things, this tasklet makes a note of how many interrupts have arrived
since it was last called. A device such as short can generate a great many interrupts in
a brief period, so it is not uncommon for several to arrive before the bottom half is
executed. Drivers must always be prepared for this possibility and must be able to
determine how much work there is to perform from the information left by the top
half.
Workqueues
Recall that workqueues invoke a function at some future time in the context of a special
worker process. Since the workqueue function runs in process context, it can
sleep if need be. You cannot, however, copy data into user space from a workqueue,
unless you use the advanced techniques we demonstrate in Chapter 15; the worker
process does not have access to any other process's address space.
The short driver, if loaded with the wq option set to a nonzero value, uses a workqueue
for its bottom-half processing. It uses the system default workqueue, so there
is no special setup code required; if your driver has special latency requirements (or
might sleep for a long time in the workqueue function), you may want to create your
own, dedicated workqueue. We do need a work_struct structure, which is declared
and initialized with the following:
static struct work_struct short_wq;
/* this line is in short_init( ) */
INIT_WORK(&short_wq, (void (*)(void *)) short_do_tasklet, NULL);
PAGE 278
Our worker function is short_do_tasklet, which we have already seen in the previous
section.
When working with a workqueue, short establishes yet another interrupt handler
that looks like this:
irqreturn_t short_wq_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
/* Grab the current time information. */
do_gettimeofday((struct timeval *) tv_head);
short_incr_tv(&tv_head);
/* Queue the bh. Don't worry about multiple enqueueing */
schedule_work(&short_wq);
short_wq_count++; /* record that an interrupt arrived */
return IRQ_HANDLED;
}
As you can see, the interrupt handler looks very much like the tasklet version, with
the exception that it calls schedule_work to arrange the bottom-half processing.
Interrupt Sharing
The notion of an IRQ conflict is almost synonymous with the PC architecture. In the
past, IRQ lines on the PC have not been able to serve more than one device, and
there have never been enough of them. As a result, frustrated users have often spent
much time with their computer case open, trying to find a way to make all of their
peripherals play well together.
Modern hardware, of course, has been designed to allow the sharing of interrupts;
the PCI bus requires it. Therefore, the Linux kernel supports interrupt sharing on all
buses, even those (such as the ISA bus) where sharing has traditionally not been supported.
Device drivers for the 2.6 kernel should be written to work with shared interrupts
if the target hardware can support that mode of operation. Fortunately,
working with shared interrupts is easy, most of the time.
Installing a Shared Handler
Shared interrupts are installed through request_irq just like nonshared ones, but
there are two differences:
- The SA_SHIRQ bit must be specified in the flags
argument when requesting the interrupt.
- The dev_id argument must be unique. Any pointer into the module's address
space will do, but dev_id definitely cannot be set to NULL.
PAGE 279
The kernel keeps a list of shared handlers associated with the interrupt, and dev_id
can be thought of as the signature that differentiates between them. If two drivers
were to register NULL as their signature on the same interrupt, things might get mixed
up at unload time, causing the kernel to oops when an interrupt arrived. For this reason,
modern kernels complain loudly if passed a NULL dev_id when registering shared
interrupts. When a shared interrupt is requested, request_irq succeeds if one of the
following is true:
- The interrupt line is free.
- All handlers already registered for that line have also specified that the IRQ is to be shared.
Whenever two or more drivers are sharing an interrupt line and the hardware interrupts
the processor on that line, the kernel invokes every handler registered for that
interrupt, passing each its own dev_id. Therefore, a shared handler must be able to
recognize its own interrupts and should quickly exit when its own device has not
interrupted. Be sure to return IRQ_NONE whenever your handler is called and finds
that the device is not interrupting.
If you need to probe for your device before requesting the IRQ line, the kernel can't
help you. No probing function is available for shared handlers. The standard probing
mechanism works if the line being used is free, but if the line is already held by
another driver with sharing capabilities, the probe fails, even if your driver would
have worked perfectly. Fortunately, most hardware designed for interrupt sharing is
also able to tell the processor which interrupt it is using, thus eliminating the need
for explicit probing.
Releasing the handler is performed in the normal way, using free_irq. Here the dev_id
argument is used to select the correct handler to release from the list of shared handlers
for the interrupt. That's why the dev_id pointer must be unique.
A driver using a shared handler needs to be careful about one more thing: it can't
play with enable_irq or disable_irq. If it does, things might go haywire for other
devices sharing the line; disabling another device's interrupts for even a short time
may create latencies that are problematic for that device and it's user. Generally, the
programmer must remember that his driver doesn't own the IRQ, and its behavior
should be more "social" than is necessary if one owns the interrupt line.
Running the Handler
As suggested earlier, when the kernel receives an interrupt, all the registered handlers
are invoked. A shared handler must be able to distinguish between interrupts
that it needs to handle and interrupts generated by other devices.
PAGE 280
Loading short with the option shared=1 installs the following handler instead of the
default:
irqreturn_t short_sh_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
int value, written;
struct timeval tv;
/* If it wasn't short, return immediately */
value = inb(short_base);
if (!(value & 0x80))
return IRQ_NONE;
/* clear the interrupting bit */
outb(value & 0x7F, short_base);
/* the rest is unchanged */
do_gettimeofday(&tv);
written = sprintf((char *)short_head,"%08u.%06u\n",
(int)(tv.tv_sec % 100000000), (int)(tv.tv_usec));
short_incr_bp(&short_head, written);
wake_up_interruptible(&short_queue); /* awake any reading process */
return IRQ_HANDLED;
}
An explanation is due here. Since the parallel port has no "interrupt-pending" bit to
check, the handler uses the ACK bit for this purpose. If the bit is high, the interrupt
being reported is for short, and the handler clears the bit.
The handler resets the bit by zeroing the high bit of the parallel interface's data
port--short assumes that pins 9 and 10 are connected together. If one of the other
devices sharing the IRQ with short generates an interrupt, short sees that its own line
is still inactive and does nothing.
A full-featured driver probably splits the work into top and bottom halves, of course,
but that's easy to add and does not have any impact on the code that implements
sharing. A real driver would also likely use the dev_id argument to determine which,
of possibly many, devices might be interrupting.
Note that if you are using a printer (instead of the jumper wire) to test interrupt management
with short, this shared handler won't work as advertised, because the
printer protocol doesn't allow for sharing, and the driver can't know whether the
interrupt was from the printer.
The /proc Interface and Shared Interrupts
Installing shared handlers in the system doesn't affect /proc/stat, which doesn't even
know about handlers. However, /proc/interrupts changes slightly.
PAGE 281
All the handlers installed for the same interrupt number appear on the same line of
/proc/interrupts. The following output (from an x86_64 system) shows how shared
interrupt handlers are displayed:
CPU0
0: 892335412 XT-PIC timer
1: 453971 XT-PIC i8042
2: 0 XT-PIC cascade
5: 0 XT-PIC libata, ehci_hcd
8: 0 XT-PIC rtc
9: 0 XT-PIC acpi
10: 11365067 XT-PIC ide2, uhci_hcd, uhci_hcd, SysKonnect SK-98xx, EMU10K1
11: 4391962 XT-PIC uhci_hcd, uhci_hcd
12: 224 XT-PIC i8042
14: 2787721 XT-PIC ide0
15: 203048 XT-PIC ide1
NMI: 41234
LOC: 892193503
ERR: 102
MIS: 0
This system has several shared interrupt lines. IRQ 5 is used for the serial ATA and
IEEE 1394 controllers; IRQ 10 has several devices, including an IDE controller, two
USB controllers, an Ethernet interface, and a sound card; and IRQ 11 also is used by
two USB controllers.
Interrupt-Driven I/O
Whenever a data transfer to or from the managed hardware might be delayed for any
reason, the driver writer should implement buffering. Data buffers help to detach
data transmission and reception from the write and read system calls, and overall system
performance benefits.
A good buffering mechanism leads to interrupt-driven I/O, in which an input buffer is
filled at interrupt time and is emptied by processes that read the device; an output
buffer is filled by processes that write to the device and is emptied at interrupt time.
An example of interrupt-driven output is the implementation of /dev/shortprint.
For interrupt-driven data transfer to happen successfully, the hardware should be
able to generate interrupts with the following semantics:
- For input, the device interrupts the processor when new data has arrived and is
ready to be retrieved by the system processor. The actual actions to perform
depend on whether the device uses I/O ports, memory mapping, or DMA.
- For output, the device delivers an interrupt either when it is ready to accept new
data or to acknowledge a successful data transfer. Memory-mapped and DMA capable
devices usually generate interrupts to tell the system they are done with
the buffer.
PAGE 282
The timing relationships between a read or write and the actual arrival of data were
introduced in the section "Blocking and Nonblocking Operations" in Chapter 6.
A Write-Buffering Example
We have mentioned the shortprint driver a couple of times; now it is time to actually
take a look. This module implements a very simple, output-oriented driver for the
parallel port; it is sufficient, however, to enable the printing of files. If you chose to
test this driver out, however, remember that you must pass the printer a file in a format
it understands; not all printers respond well when given a stream of arbitrary
data.
The shortprint driver maintains a one-page circular output buffer. When a user-space
process writes data to the device, that data is fed into the buffer, but the write
method does not actually perform any I/O. Instead, the core of shortp_write looks
like this:
while (written < count) {
/* Hang out until some buffer space is available. */
space = shortp_out_space( );
if (space <= 0) {
if (wait_event_interruptible(shortp_out_queue,
(space = shortp_out_space( )) > 0))
goto out;
}
/* Move data into the buffer. */
if ((space + written) > count)
space = count - written;
if (copy_from_user((char *) shortp_out_head, buf, space)) {
up(&shortp_out_sem);
return -EFAULT;
}
shortp_incr_out_bp(&shortp_out_head, space);
buf += space;
written += space;
/* If no output is active, make it active. */
spin_lock_irqsave(&shortp_out_lock, flags);
if (! shortp_output_active)
shortp_start_output( );
spin_unlock_irqrestore(&shortp_out_lock, flags);
}
out:
*f_pos += written;
A semaphore (shortp_out_sem) controls access to the circular buffer; shortp_write
obtains that semaphore just prior to the code fragment above. While holding the semaphore,
it attempts to feed data into the circular buffer. The function shortp_out_space
returns the amount of contiguous space available (so there is no need to worry about
PAGE 283
buffer wraps); if that amount is 0, the driver waits until some space is freed. It then
copies as much data as it can into the buffer.
Once there is data to output, shortp_write must ensure that the data is written to the
device. The actual writing is done by way of a workqueue function; shortp_write
must kick that function off if it is not already running. After obtaining a separate
spinlock that controls access to variables used on the consumer side of the output
buffer (including shortp_output_active), it calls shortp_start_output if need be. Then
it's just a matter of noting how much data was "written" to the buffer and returning.
The function that starts the output process looks like the following:
static void shortp_start_output(void)
{
if (shortp_output_active) /* Should never happen */
return;
/* Set up our 'missed interrupt' timer */
shortp_output_active = 1;
shortp_timer.expires = jiffies + TIMEOUT;
add_timer(&shortp_timer);
/* And get the process going. */
queue_work(shortp_workqueue, &shortp_work);
}
The reality of dealing with hardware is that you can, occasionally, lose an interrupt
from the device. When this happens, you really do not want your driver to stop forevermore
until the system is rebooted; that is not a user-friendly way of doing things.
It is far better to realize that an interrupt has been missed, pickup the pieces, and go
on. To that end, shortprint sets a kernel timer whenever it outputs data to the device.
If the timer expires, we may have missed an interrupt. We look at the timer function
shortly, but, for the moment, let's stick with the main output functionality. That is
implemented in our workqueue function, which, as you can see above, is scheduled
here. The core of that function looks like the following:
spin_lock_irqsave(&shortp_out_lock, flags);
/* Have we written everything? */
if (shortp_out_head = = shortp_out_tail) { /* empty */
shortp_output_active = 0;
wake_up_interruptible(&shortp_empty_queue);
del_timer(&shortp_timer);
}
/* Nope, write another byte */
else
shortp_do_write( );
/* If somebody's waiting, maybe wake them up. */
if (((PAGE_SIZE + shortp_out_tail -shortp_out_head) % PAGE_SIZE) > SP_MIN_SPACE)
{
PAGE 284
wake_up_interruptible(&shortp_out_queue);
}
spin_unlock_irqrestore(&shortp_out_lock, flags);
Since we are dealing with the output side's shared variables, we must obtain the
spinlock. Then we look to see whether there is any more data to send out; if not, we
note that output is no longer active, delete the timer, and wake up anybody who
might have been waiting for the queue to become completely empty (this sort of wait
is done when the device is closed). If, instead, there remains data to write, we call
shortp_do_write to actually send a byte to the hardware.
Then, since we may have freed space in the output buffer, we consider waking up
any processes waiting to add more data to that buffer. We do not perform that
wakeup unconditionally, however; instead, we wait until a minimum amount of
space is available. There is no point in awakening a writer every time we take one
byte out of the buffer; the cost of awakening the process, scheduling it to run, and
putting it back to sleep is too high for that. Instead, we should wait until that process
is able to move a substantial amount of data into the buffer at once. This technique
is common in buffering, interrupt-driven drivers.
For completeness, here is the code that actually writes the data to the port:
static void shortp_do_write(void)
{
unsigned char cr = inb(shortp_base + SP_CONTROL);
/* Something happened; reset the timer */
mod_timer(&shortp_timer, jiffies + TIMEOUT);
/* Strobe a byte out to the device */
outb_p(*shortp_out_tail, shortp_base+SP_DATA);
shortp_incr_out_bp(&shortp_out_tail, 1);
if (shortp_delay)
udelay(shortp_delay);
outb_p(cr | SP_CR_STROBE, shortp_base+SP_CONTROL);
if (shortp_delay)
udelay(shortp_delay);
outb_p(cr & ~SP_CR_STROBE, shortp_base+SP_CONTROL);
}
Here, we reset the timer to reflect the fact that we have made some progress, strobe
the byte out to the device, and update the circular buffer pointer.
The workqueue function does not resubmit itself directly, so only a single byte will be
written to the device. At some point, the printer will, in its slow way, consume the
byte and become ready for the next one; it will then interrupt the processor. The
interrupt handler used in shortprint is short and simple:
static irqreturn_t shortp_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
if (! shortp_output_active)
return IRQ_NONE;
PAGE 285
/* Remember the time, and farm off the rest to the workqueue function */
do_gettimeofday(&shortp_tv);
queue_work(shortp_workqueue, &shortp_work);
return IRQ_HANDLED;
}
Since the parallel port does not require an explicit interrupt acknowledgment, all the
interrupt handler really needs to do is to tell the kernel to run the workqueue function
again.
What if the interrupt never comes? The driver code that we have seen thus far would
simply come to a halt. To keep that from happening, we set a timer back a few pages
ago. The function that is executed when that timer expires is:
static void shortp_timeout(unsigned long unused)
{
unsigned long flags;
unsigned char status;
if (! shortp_output_active)
return;
spin_lock_irqsave(&shortp_out_lock, flags);
status = inb(shortp_base + SP_STATUS);
/* If the printer is still busy we just reset the timer */
if ((status & SP_SR_BUSY) = = 0 || (status & SP_SR_ACK)) {
shortp_timer.expires = jiffies + TIMEOUT;
add_timer(&shortp_timer);
spin_unlock_irqrestore(&shortp_out_lock, flags);
return;
}
/* Otherwise we must have dropped an interrupt. */
spin_unlock_irqrestore(&shortp_out_lock, flags);
shortp_interrupt(shortp_irq, NULL, NULL);
}
If no output is supposed to be active, the timer function simply returns; this keeps
the timer from resubmitting itself when things are being shut down. Then, after taking
the lock, we query the status of the port; if it claims to be busy, it simply hasn't
gotten around to interrupting us yet, so we reset the timer and return. Printers can, at
times, take a very long time to make themselves ready; consider the printer that runs
out of paper while everybody is gone over a long weekend. In such situations, there
is nothing to do other than to wait patiently until something changes.
If, however, the printer claims to be ready, we must have missed its interrupt. In that
case, we simply invoke our interrupt handler manually to get the output process
moving again.
The shortprint driver does not support reading from the port; instead, it behaves like
shortint and returns interrupt timing information. The implementation of an interrupt-driven
read method would be very similar to what we have seen, however. Data
PAGE 286
from the device would be read into a driver buffer; it would be copied out to user
space only when a significant amount of data has accumulated in the buffer, the full
read request has been satisfied, or some sort of timeout occurs.
Quick Reference
These symbols related to interrupt management were introduced in this chapter:
#include <linux/interrupt.h>
int request_irq(unsigned int irq, irqreturn_t (*handler)( ), unsigned long flags, const char *dev_name, void *dev_id);
void free_irq(unsigned int irq, void *dev_id);
Calls that register and unregister an interrupt handler.
#include <linux/irq.h.h>
int can_request_irq(unsigned int irq, unsigned long flags);
This function, available on the i386 and x86_64 architectures, returns a nonzero
value if an attempt to allocate the given interrupt line succeeds.
#include <asm/signal.h>
SA_INTERRUPT
SA_SHIRQ
SA_SAMPLE_RANDOM
Flags for request_irq. SA_INTERRUPT requests installation of a fast handler (as
opposed to a slow one). SA_SHIRQ installs a shared handler, and the third flag
asserts that interrupt timestamps can be used to generate system entropy.
/proc/interrupts
/proc/stat
Filesystem nodes that report information about hardware interrupts and
installed handlers.
unsigned long probe_irq_on(void);
int probe_irq_off(unsigned long);
Functions used by the driver when it has to probe to determine which interrupt
line is being used by a device. The result of probe_irq_on must be passed back to
probe_irq_off after the interrupt has been generated. The return value of probe_
irq_off is the detected interrupt number.
IRQ_NONE
IRQ_HANDLED
IRQ_RETVAL(int x)
The possible return values from an interrupt handler, indicating whether an
actual interrupt from the device was present.
PAGE 287
void disable_irq(int irq);
void disable_irq_nosync(int irq);
void enable_irq(int irq);
A driver can enable and disable interrupt reporting. If the hardware tries to generate
an interrupt while interrupts are disabled, the interrupt is lost forever. A
driver using a shared handler must not use these functions.
void local_irq_save(unsigned long flags);
void local_irq_restore(unsigned long flags);
Use local_irq_save to disable interrupts on the local processor and remember
their previous state. The flags can be passed to local_irq_restore to restore the
previous interrupt state.
void local_irq_disable(void);
void local_irq_enable(void);
Functions that unconditionally disable and enable interrupts on the current
processor.
PAGE 288
CHAPTER 11
Data Types in the Kernel
Before we go on to more advanced topics, we need to stop for a quick note on portability
issues. Modern versions of the Linux kernel are highly portable, running on
numerous different architectures. Given the multiplatform nature of Linux, drivers
intended for serious use should be portable as well.
But a core issue with kernel code is being able both to access data items of known
length (for example, filesystem data structures or registers on device boards) and to
exploit the capabilities of different processors (32-bit and 64-bit architectures, and
possibly 16 bit as well).
Several of the problems encountered by kernel developers while porting x86 code to
new architectures have been related to incorrect data typing. Adherence to strict data
typing and compiling with the -Wall -Wstrict-prototypes flags can prevent most bugs.
Data types used by kernel data are divided into three main classes: standard C types
such as int, explicitly sized types such as u32, and types used for specific kernel
objects, such as pid_t. We are going to see when and how each of the three typing
classes should be used. The final sections of the chapter talk about some other typical
problems you might run into when porting driver code from the x86 to other
platforms, and introduce the generalized support for linked lists exported by recent
kernel headers.
If you follow the guidelines we provide, your driver should compile and run even on
platforms on which you are unable to test it.
Use of Standard C Types
Although most programmers are accustomed to freely using standard types like int
and long, writing device drivers requires some care to avoid typing conflicts and
obscure bugs.
The problem is that you can't use the standard types when you need "a 2-byte filler"
or "something representing a 4-byte string," because the normal C data types are not
PAGE 289
the same size on all architectures. To show the data size of the various C types, the
datasize program has been included in the sample files provided on O'Reilly's FTP
site in the directory misc-progs. This is a sample run of the program on an i386 system
(the last four types shown are introduced in the next section):
morgana% misc-progs/datasize
arch Size: char short int long ptr long-long u8 u16 u32 u64
i686 1 2 4 4 4 8 1 2 4 8
The program can be used to show that long integers and pointers feature a different
size on 64-bit platforms, as demonstrated by running the program on different Linux
computers:
arch Size: char short int long ptr long-long u8 u16 u32 u64
i386 1 2 4 4 4 8 1 2 4 8
alpha 1 2 4 8 8 8 1 2 4 8
armv4l 1 2 4 4 4 8 1 2 4 8
ia64 1 2 4 8 8 8 1 2 4 8
m68k 1 2 4 4 4 8 1 2 4 8
mips 1 2 4 4 4 8 1 2 4 8
ppc 1 2 4 4 4 8 1 2 4 8
sparc 1 2 4 4 4 8 1 2 4 8
sparc64 1 2 4 4 4 8 1 2 4 8
x86_64 1 2 4 8 8 8 1 2 4 8
It's interesting to note that the SPARC 64 architecture runs with a 32-bit user space,
so pointers are 32 bits wide there, even though they are 64 bits wide in kernel space.
This can be verified by loading the kdatasize module (available in the directory miscmodules
within the sample files). The module reports size information at load time
using printk and returns an error (so there's no need to unload it):
kernel: arch Size: char short int long ptr long-long u8 u16 u32 u64
kernel: sparc64 1 2 4 8 8 8 1 2 4 8
Although you must be careful when mixing different data types, sometimes there are
good reasons to do so. One such situation is for memory addresses, which are special
as far as the kernel is concerned. Although, conceptually, addresses are pointers,
memory administration is often better accomplished by using an unsigned integer
type; the kernel treats physical memory like a huge array, and a memory address is
just an index into the array. Furthermore, a pointer is easily dereferenced; when dealing
directly with memory addresses, you almost never want to dereference them in
this manner. Using an integer type prevents this dereferencing, thus avoiding bugs.
Therefore, generic memory addresses in the kernel are usually unsigned long, exploiting
the fact that pointers and long integers are always the same size, at least on all the
platforms currently supported by Linux.
For what it's worth, the C99 standard defines the intptr_t and uintptr_t types for
an integer variable that can hold a pointer value. These types are almost unused in
the 2.6 kernel, however.
PAGE 290
Assigning an Explicit Size to Data Items
Sometimes kernel code requires data items of a specific size, perhaps to match predefined
binary structures,* to communicate with user space, or to align data within
structures by inserting "padding" fields (but refer to the section "Data Alignment"
for information about alignment issues).
The kernel offers the following data types to use whenever you need to know the size
of your data. All the types are declared in <asm/types.h>, which, in turn, is included
by <linux/types.h>:
u8; /* unsigned byte (8 bits) */
u16; /* unsigned word (16 bits) */
u32; /* unsigned 32-bit value */
u64; /* unsigned 64-bit value */
The corresponding signed types exist, but are rarely needed; just replace u with s in
the name if you need them.
If a user-space program needs to use these types, it can prefix the names with a double
underscore: __u8 and the other types are defined independent of __KERNEL__. If,
for example, a driver needs to exchange binary structures with a program running in
user space by means of ioctl, the header files should declare 32-bit fields in the structures
as __u32.
It's important to remember that these types are Linux specific, and using them hinders
porting software to other Unix flavors. Systems with recent compilers support
the C99-standard types, such as uint8_t and uint32_t; if portability is a concern,
those types can be used in favor of the Linux-specific variety.
You might also note that sometimes the kernel uses conventional types, such as
unsigned int, for items whose dimension is architecture independent. This is usually
done for backward compatibility. When u32 and friends were introduced in Version
1.1.67, the developers couldn't change existing data structures to the new types
because the compiler issues a warning when there is a type mismatch between the
structure field and the value being assigned to it. Linus didn't expect the operating
system (OS) he wrote for his own use to become multiplatform; as a result, old structures
are sometimes loosely typed.
* This happens when reading partition tables, when executing a binary file, or when decoding a network packet.
As a matter of fact, the compiler signals type inconsistencies even if the two types are just different
names for the same object, such as unsigned long and u32 on the PC.
PAGE 291
Interface-Specific Types
Some of the commonly used data types in the kernel have their own typedef statements,
thus preventing any portability problems. For example, a process identifier
(pid) is usually pid_t instead of int. Using pid_t masks any possible difference in the
actual data typing. We use the expression interface-specific to refer to a type defined
by a library in order to provide an interface to a specific data structure.
Note that, in recent times, relatively few new interface-specific types have been
defined. Use of the typedef statement has gone out of favor among many kernel
developers, who would rather see the real type information used directly in the code,
rather than hidden behind a user-defined type. Many older interface-specific types
remain in the kernel, however, and they will not be going away anytime soon.
Even when no interface-specific type is defined, it's always important to use the
proper data type in a way consistent with the rest of the kernel. A jiffy count, for
instance, is always unsigned long, independent of its actual size, so the unsigned long
type should always be used when working with jiffies. In this section we concentrate
on use of _t types.
Many _t types are defined in <linux/types.h>, but the list is rarely useful. When you
need a specific type, you'll find it in the prototype of the functions you need to call or
in the data structures you use.
Whenever your driver uses functions that require such "custom" types and you don't
follow the convention, the compiler issues a warning; if you use the -Wall compiler
flag and are careful to remove all the warnings, you can feel confident that your code
is portable.
The main problem with _t data items is that when you need to print them, it's not
always easy to choose the right printk or printf format, and warnings you resolve on
one architecture reappear on another. For example, how would you print a size_t,
that is unsigned long on some platforms and unsigned int on some others?
Whenever you need to print some interface-specific data, the best way to do it is by
casting the value to the biggest possible type (usually long or unsigned long) and then
printing it through the corresponding format. This kind of tweaking won't generate
errors or warnings because the format matches the type, and you won't lose data bits
because the cast is either a null operation or an extension of the item to a bigger data
type.
In practice, the data items we're talking about aren't usually meant to be printed, so
the issue applies only to debugging messages. Most often, the code needs only to
store and compare the interface-specific types, in addition to passing them as arguments
to library or kernel functions.
Although _t types are the correct solution for most situations, sometimes the right type
doesn't exist. This happens for some old interfaces that haven't yet been cleaned up.
PAGE 292
The one ambiguous point we've found in the kernel headers is data typing for I/O
functions, which is loosely defined (see the section "Platform Dependencies" in
Chapter 9). The loose typing is mainly there for historical reasons, but it can create
problems when writing code. For example, one can get into trouble by swapping the
arguments to functions like outb; if there were a port_t type, the compiler would find
this type of error.
Other Portability Issues
In addition to data typing, there are a few other software issues to keep in mind
when writing a driver if you want it to be portable across Linux platforms.
A general rule is to be suspicious of explicit constant values. Usually the code has
been parameterized using preprocessor macros. This section lists the most important
portability problems. Whenever you encounter other values that have been
parameterized, you can find hints in the header files and in the device drivers distributed
with the official kernel.
Time Intervals
When dealing with time intervals, don't assume that there are 1000 jiffies per second.
Although this is currently true for the i386 architecture, not every Linux platform
runs at this speed. The assumption can be false even for the x86 if you play
with the HZ value (as some people do), and nobody knows what will happen in future
kernels. Whenever you calculate time intervals using jiffies, scale your times using HZ
(the number of timer interrupts per second). For example, to check against a timeout
of half a second, compare the elapsed time against HZ/2. More generally, the
number of jiffies corresponding to msec milliseconds is always msec*HZ/1000.
Page Size
When playing games with memory, remember that a memory page is PAGE_SIZE
bytes, not 4 KB. Assuming that the page size is 4 KB and hard-coding the value is a
common error among PC programmers, instead, supported platforms show page
sizes from 4 KB to 64 KB, and sometimes they differ between different implementations
of the same platform. The relevant macros are PAGE_SIZE and PAGE_SHIFT. The
latter contains the number of bits to shift an address to get its page number. The
number currently is 12 or greater for pages that are 4 KB and larger. The macros are
defined in <asm/page.h>; user-space programs can use the getpagesize library function
if they ever need the information.
PAGE 293
Let's look at a nontrivial situation. If a driver needs 16 KB for temporary data, it
shouldn't specify an order of 2 to get_free_pages. You need a portable solution. Such a
solution, fortunately, has been written by the kernel developers and is called get_order:
#include <asm/page.h>
int order = get_order(16*1024);
buf = get_free_pages(GFP_KERNEL, order);
Remember that the argument to get_order must be a power of two.
Byte Order
Be careful not to make assumptions about byte ordering. Whereas the PC stores
multibyte values low-byte first (little end first, thus little-endian), some high-level
platforms work the other way (big-endian). Whenever possible, your code should be
written such that it does not care about byte ordering in the data it manipulates.
However, sometimes a driver needs to build an integer number out of single bytes or
do the opposite, or it must communicate with a device that expects a specific order.
The include file <asm/byteorder.h> defines either __BIG_ENDIAN or __LITTLE_ENDIAN,
depending on the processor's byte ordering. When dealing with byte ordering issues,
you could code a bunch of #ifdef __LITTLE_ENDIAN conditionals, but there is a better
way. The Linux kernel defines a set of macros that handle conversions between
the processor's byte ordering and that of the data you need to store or load in a specific
byte order. For example:
u32 cpu_to_le32 (u32);
u32 le32_to_cpu (u32);
These two macros convert a value from whatever the CPU uses to an unsigned, littleendian,
32-bit quantity and back. They work whether your CPU is big-endian or little-endian
and, for that matter, whether it is a 32-bit processor or not. They return
their argument unchanged in cases where there is no work to be done. Use of these
macros makes it easy to write portable code without having to use a lot of conditional
compilation constructs.
There are dozens of similar routines; you can see the full list in <linux/byteorder/
big_endian.h> and <linux/byteorder/little_endian.h>. After a while, the pattern is
not hard to follow. be64_to_cpu converts an unsigned, big-endian, 64-bit value to
the internal CPU representation. le16_to_cpus, instead, handles signed, littleendian,
16-bit quantities. When dealing with pointers, you can also use functions
like cpu_to_le32p, which take a pointer to the value to be converted rather than the
value itself. See the include file for the rest.
Data Alignment
The last problem worth considering when writing portable code is how to access
unaligned data--for example, how to read a 4-byte value stored at an address that
PAGE 294
isn't a multiple of 4 bytes. i386 users often access unaligned data items, but not all
architectures permit it. Many modern architectures generate an exception every time
the program tries unaligned data transfers; data transfer is handled by the exception
handler, with a great performance penalty. If you need to access unaligned data, you
should use the following macros:
#include <asm/unaligned.h>
get_unaligned(ptr);
put_unaligned(val, ptr);
These macros are typeless and work for every data item, whether it's one, two, four,
or eight bytes long. They are defined with any kernel version.
Another issue related to alignment is portability of data structures across platforms.
The same data structure (as defined in the C-language source file) can be compiled
differently on different platforms. The compiler arranges structure fields to be
aligned according to conventions that differ from platform to platform.
In order to write data structures for data items that can be moved across architectures,
you should always enforce natural alignment of the data items in addition to
standardizing on a specific endianness. Natural alignment means storing data items
at an address that is a multiple of their size (for instance, 8-byte items go in an
address multiple of 8). To enforce natural alignment while preventing the compiler
to arrange the fields in unpredictable ways, you should use filler fields that avoid
leaving holes in the data structure.
To show how alignment is enforced by the compiler, the dataalign program is distributed
in the misc-progs directory of the sample code, and an equivalent kdataalign
module is part of misc-modules. This is the output of the program on several platforms
and the output of the module on the SPARC64:
arch Align: char short int long ptr long-long u8 u16 u32 u64
i386 1 2 4 4 4 4 1 2 4 4
i686 1 2 4 4 4 4 1 2 4 4
alpha 1 2 4 8 8 8 1 2 4 8
armv4l 1 2 4 4 4 4 1 2 4 4
ia64 1 2 4 8 8 8 1 2 4 8
mips 1 2 4 4 4 8 1 2 4 8
ppc 1 2 4 4 4 8 1 2 4 8
sparc 1 2 4 4 4 8 1 2 4 8
sparc64 1 2 4 4 4 8 1 2 4 8
x86_64 1 2 4 8 8 8 1 2 4 8
kernel: arch Align: char short int long ptr long-long u8 u16 u32 u64
kernel: sparc64 1 2 4 8 8 8 1 2 4 8
It's interesting to note that not all platforms align 64-bit values on 64-bit boundaries,
so you need filler fields to enforce alignment and ensure portability.
Finally, be aware that the compiler may quietly insert padding into structures itself to
ensure that every field is aligned for good performance on the target processor. If you
PAGE 295
are defining a structure that is intended to match a structure expected by a device,
this automatic padding may thwart your attempt. The way around this problem is to
tell the compiler that the structure must be "packed," with no fillers added. For
example, the kernel header file <linux/edd.h> defines several data structures used in
interfacing with the x86 BIOS, and it includes the following definition:
struct {
u16 id;
u64 lun;
u16 reserved1;
u32 reserved2;
} __attribute__ ((packed)) scsi;
Without the __attribute__ ((packed)), the lun field would be preceded by two filler
bytes or six if we compile the structure on a 64-bit platform.
Pointers and Error Values
Many internal kernel functions return a pointer value to the caller. Many of those
functions can also fail. In most cases, failure is indicated by returning a NULL pointer
value. This technique works, but it is unable to communicate the exact nature of the
problem. Some interfaces really need to return an actual error code so that the caller
can make the right decision based on what actually went wrong.
A number of kernel interfaces return this information by encoding the error code in a
pointer value. Such functions must be used with care, since their return value cannot
simply be compared against NULL. To help in the creation and use of this sort of interface,
a small set of functions has been made available (in <linux/err.h>).
A function returning a pointer type can return an error value with:
void *ERR_PTR(long error);
where error is the usual negative error code. The caller can use IS_ERR to test
whether a returned pointer is an error code or not:
long IS_ERR(const void *ptr);
If you need the actual error code, it can be extracted with:
long PTR_ERR(const void *ptr);
You should use PTR_ERR only on a value for which IS_ERR returns a true value;
any other value is a valid pointer.
Linked Lists
Operating system kernels, like many other programs, often need to maintain lists of
data structures. The Linux kernel has, at times, been host to several linked list implementations
at the same time. To reduce the amount of duplicated code, the kernel
PAGE 296
developers have created a standard implementation of circular, doubly linked lists;
others needing to manipulate lists are encouraged to use this facility.
When working with the linked list interface, you should always bear in mind that the
list functions perform no locking. If there is a possibility that your driver could
attempt to perform concurrent operations on the same list, it is your responsibility to
implement a locking scheme. The alternatives (corrupted list structures, data loss,
kernel panics) tend to be difficult to diagnose.
To use the list mechanism, your driver must include the file <linux/list.h>. This file
defines a simple structure of type list_head:
struct list_head {
struct list_head *next, *prev;
};
Linked lists used in real code are almost invariably made up of some type of structure,
each one describing one entry in the list. To use the Linux list facility in your
code, you need only embed a list_head inside the structures that make up the list. If
your driver maintains a list of things to do, say, its declaration would look something
like this:
struct todo_struct {
struct list_head list;
int priority; /* driver specific */
/* ... add other driver-specific fields */
};
The head of the list is usually a stand alone list_head structure. Figure 11-1 shows
how the simple struct list_head is used to maintain a list of data structures.
Figure 11-1. The list_head data structure
PAGE 297
List heads must be initialized prior to use with the INIT_LIST_HEAD macro. A "things
to do" list head could be declared and initialized with:
struct list_head todo_list;
INIT_LIST_HEAD(&todo_list);
Alternatively, lists can be initialized at compile time:
LIST_HEAD(todo_list);
Several functions are defined in <linux/list.h> that work with lists:
list_add(struct list_head *new, struct list_head *head);
Adds the new entry immediately after the list head--normally at the beginning of
the list. Therefore, it can be used to build stacks. Note, however, that the head
need not be the nominal head of the list; if you pass a list_head structure that
happens to be in the middle of the list somewhere, the new entry goes immediately
after it. Since Linux lists are circular, the head of the list is not generally different
from any other entry.
list_add_tail(struct list_head *new, struct list_head *head);
Adds a new entry just before the given list head--at the end of the list, in other
words. list_add_tail can, thus, be used to build first-in first-out queues.
list_del(struct list_head *entry);
list_del_init(struct list_head *entry);
The given entry is removed from the list. If the entry might ever be reinserted
into another list, you should use list_del_init, which reinitializes the linked list
pointers.
list_move(struct list_head *entry, struct list_head *head);
list_move_tail(struct list_head *entry, struct list_head *head);
The given entry is removed from its current list and added to the beginning of
head. To put the entry at the end of the new list, use list_move_tail instead.
list_empty(struct list_head *head);
Returns a nonzero value if the given list is empty.
list_splice(struct list_head *list, struct list_head *head);
Joins two lists by inserting list immediately after head.
The list_head structures are good for implementing a list of like structures, but the
invoking program is usually more interested in the larger structures that make up the
list as a whole. A macro, list_entry, is provided that maps a list_head structure
pointer back into a pointer to the structure that contains it. It is invoked as follows:
list_entry(struct list_head *ptr, type_of_struct, field_name);
where ptr is a pointer to the struct list_head being used, type_of_struct is the type
of the structure containing the ptr, and field_name is the name of the list field within
PAGE 298
the structure. In our todo_struct structure from before, the list field is called simply
list. Thus, we would turn a list entry into its containing structure with a line such as:
struct todo_struct *todo_ptr =
list_entry(listptr, struct todo_struct, list);
The list_entry macro takes a little getting used to but is not that hard to use.
The traversal of linked lists is easy: one need only follow the prev and next pointers.
As an example, suppose we want to keep the list of todo_struct items sorted in
descending priority order. A function to add a new entry would look something like
this:
void todo_add_entry(struct todo_struct *new)
{
struct list_head *ptr;
struct todo_struct *entry;
for (ptr = todo_list.next; ptr != &todo_list; ptr = ptr->next) {
entry = list_entry(ptr, struct todo_struct, list);
if (entry->priority < new->priority) {
list_add_tail(&new->list, ptr);
return;
}
}
list_add_tail(&new->list, &todo_struct)
}
However, as a general rule, it is better to use one of a set of predefined macros for
creating loops that iterate through lists. The previous loop, for example, could be
coded as:
void todo_add_entry(struct todo_struct *new)
{
struct list_head *ptr;
struct todo_struct *entry;
list_for_each(ptr, &todo_list) {
entry = list_entry(ptr, struct todo_struct, list);
if (entry->priority < new->priority) {
list_add_tail(&new->list, ptr);
return;
}
}
list_add_tail(&new->list, &todo_struct)
}
PAGE 299
Using the provided macros helps avoid simple programming errors; the developers of
these macros have also put some effort into ensuring that they perform well. A few
variants exist:
list_for_each(struct list_head *cursor, struct list_head *list)
This macro creates a for loop that executes once with cursor pointing at each
successive entry in the list. Be careful about changing the list while iterating
through it.
list_for_each_prev(struct list_head *cursor, struct list_head *list)
This version iterates backward through the list.
list_for_each_safe(struct list_head *cursor, struct list_head *next, struct list_head *list)
If your loop may delete entries in the list, use this version. It simply stores the
next entry in the list in next at the beginning of the loop, so it does not get confused
if the entry pointed to by cursor is deleted.
list_for_each_entry(type *cursor, struct list_head *list, member)
list_for_each_entry_safe(type *cursor, type *next, struct list_head *list, member)
These macros ease the process of dealing with a list containing a given type of
structure. Here, cursor is a pointer to the containing structure type, and member
is the name of the list_head structure within the containing structure. With
these macros, there is no need to put list_entry calls inside the loop.
If you look inside <linux/list.h>, you see some additional declarations. The hlist
type is a doubly linked list with a separate, single-pointer list head type; it is often
used for creation of hash tables and similar structures. There are also macros for iterating
through both types of lists that are intended to work with the read-copy-update
mechanism (described in the section "Read-Copy-Update" in Chapter 5). These
primitives are unlikely to be useful in device drivers; see the header file if you would
like more information on how they work.
Quick Reference
The following symbols were introduced in this chapter:
#include <linux/types.h>
typedef u8;
typedef u16;
typedef u32;
typedef u64;
Types guaranteed to be 8-, 16-, 32-, and 64-bit unsigned integer values. The
equivalent signed types exist as well. In user space, you can refer to the types
as __u8, __u16, and so forth.
PAGE 300
#include <asm/page.h>
PAGE_SIZE
PAGE_SHIFT
Symbols that define the number of bytes per page for the current architecture
and the number of bits in the page offset (12 for 4-KB pages and 13 for 8-KB
pages).
#include <asm/byteorder.h>
__LITTLE_ENDIAN
__BIG_ENDIAN
Only one of the two symbols is defined, depending on the architecture.
#include <asm/byteorder.h>
u32 __cpu_to_le32 (u32);
u32 __le32_to_cpu (u32);
Functions that convert between known byte orders and that of the processor.
There are more than 60 such functions; see the various files in include/linux/
byteorder/ for a full list and the ways in which they are defined.
#include <asm/unaligned.h>
get_unaligned(ptr);
put_unaligned(val, ptr);
Some architectures need to protect unaligned data access using these macros.
The macros expand to normal pointer dereferencing for architectures that permit
you to access unaligned data.
#include <linux/err.h>
void *ERR_PTR(long error);
long PTR_ERR(const void *ptr);
long IS_ERR(const void *ptr);
Functions allow error codes to be returned by functions that return a pointer
value.
#include <linux/list.h>
list_add(struct list_head *new, struct list_head *head);
list_add_tail(struct list_head *new, struct list_head *head);
list_del(struct list_head *entry);
list_del_init(struct list_head *entry);
list_empty(struct list_head *head);
list_entry(entry, type, member);
list_move(struct list_head *entry, struct list_head *head);
list_move_tail(struct list_head *entry, struct list_head *head);
list_splice(struct list_head *list, struct list_head *head);
Functions that manipulate circular, doubly linked lists.
PAGE 301
list_for_each(struct list_head *cursor, struct list_head *list)
list_for_each_prev(struct list_head *cursor, struct list_head *list)
list_for_each_safe(struct list_head *cursor, struct list_head *next, struct list_head *list)
list_for_each_entry(type *cursor, struct list_head *list, member)
list_for_each_entry_safe(type *cursor, type *next struct list_head *list, member)
Convenience macros for iterating through linked lists.
PAGE 302
CHAPTER 12
PCI Drivers
While Chapter 9 introduced the lowest levels of hardware control, this chapter provides
an overview of the higher-level bus architectures. A bus is made up of both an
electrical interface and a programming interface. In this chapter, we deal with the
programming interface.
This chapter covers a number of bus architectures. However, the primary focus is on
the kernel functions that access Peripheral Component Interconnect (PCI) peripherals,
because these days the PCI bus is the most commonly used peripheral bus on
desktops and bigger computers. The bus is the one that is best supported by the kernel.
ISA is still common for electronic hobbyists and is described later, although it is
pretty much a bare-metal kind of bus, and there isn't much to say in addition to
what is covered in Chapters 9 and 10.
The PCI Interface
Although many computer users think of PCI as a way of laying out electrical wires, it
is actually a complete set of specifications defining how different parts of a computer
should interact.
The PCI specification covers most issues related to computer interfaces. We are not
going to cover it all here; in this section, we are mainly concerned with how a PCI
driver can find its hardware and gain access to it. The probing techniques discussed
in the sections "Module Parameters" in Chapter 2 and "Auto-Detecting the IRQ
Number" in Chapter 10 can be used with PCI devices, but the specification offers an
alternative that is preferable to probing.
The PCI architecture was designed as a replacement for the ISA standard, with three
main goals: to get better performance when transferring data between the computer
and its peripherals, to be as platform independent as possible, and to simplify adding
and removing peripherals to the system.
PAGE 303
The PCI bus achieves better performance by using a higher clock rate than ISA; its
clock runs at 25 or 33 MHz (its actual rate being a factor of the system clock), and
66-MHz and even 133-MHz implementations have recently been deployed as well.
Moreover, it is equipped with a 32-bit data bus, and a 64-bit extension has been
included in the specification. Platform independence is often a goal in the design of a
computer bus, and it's an especially important feature of PCI, because the PC world
has always been dominated by processor-specific interface standards. PCI is currently
used extensively on IA-32, Alpha, PowerPC, SPARC64, and IA-64 systems,
and some other platforms as well.
What is most relevant to the driver writer, however, is PCI's support for auto-detection
of interface boards. PCI devices are jumperless (unlike most older peripherals)
and are automatically configured at boot time. Then, the device driver must be able
to access configuration information in the device in order to complete initialization.
This happens without the need to perform any probing.
PCI Addressing
Each PCI peripheral is identified by a bus number, a device number, and a function
number. The PCI specification permits a single system to host up to 256 buses, but
because 256 buses are not sufficient for many large systems, Linux now supports PCI
domains. Each PCI domain can host up to 256 buses. Each bus hosts up to 32
devices, and each device can be a multifunction board (such as an audio device with
an accompanying CD-ROM drive) with a maximum of eight functions. Therefore,
each function can be identified at hardware level by a 16-bit address, or key. Device
drivers written for Linux, though, don't need to deal with those binary addresses,
because they use a specific data structure, called pci_dev, to act on the devices.
Most recent workstations feature at least two PCI buses. Plugging more than one bus
in a single system is accomplished by means of bridges, special-purpose PCI peripherals
whose task is joining two buses. The overall layout of a PCI system is a tree where
each bus is connected to an upper-layer bus, up to bus 0 at the root of the tree. The
CardBus PC-card system is also connected to the PCI system via bridges. A typical
PCI system is represented in Figure 12-1, where the various bridges are highlighted.
The 16-bit hardware addresses associated with PCI peripherals, although mostly hidden
in the struct pci_dev object, are still visible occasionally, especially when lists of
devices are being used. One such situation is the output of lspci (part of the pciutils
package, available with most distributions) and the layout of information in /proc/pci
and /proc/bus/pci. The sysfs representation of PCI devices also shows this addressing
scheme, with the addition of the PCI domain information.* When the hardware
address is displayed, it can be shown as two values (an 8-bit bus number and an 8-bit
* Some architectures also display the PCI domain information in the /proc/pci and /proc/bus/pci files.
PAGE 304
Figure 12-1. Layout of a typical PCI system
device and function number), as three values (bus, device, and function), or as four
values (domain, bus, device, and function); all the values are usually displayed in
hexadecimal.
For example, /proc/bus/pci/devices uses a single 16-bit field (to ease parsing and sorting),
while /proc/bus/busnumber splits the address into three fields. The following
shows how those addresses appear, showing only the beginning of the output lines:
$ lspci | cut -d: -f1-3
0000:00:00.0 Host bridge
0000:00:00.1 RAM memory
0000:00:00.2 RAM memory
0000:00:02.0 USB Controller
0000:00:04.0 Multimedia audio controller
0000:00:06.0 Bridge
0000:00:07.0 ISA bridge
0000:00:09.0 USB Controller
0000:00:09.1 USB Controller
0000:00:09.2 USB Controller
0000:00:0c.0 CardBus bridge
0000:00:0f.0 IDE interface
0000:00:10.0 Ethernet controller
0000:00:12.0 Network controller
0000:00:13.0 FireWire (IEEE 1394)
0000:00:14.0 VGA compatible controller
$ cat /proc/bus/pci/devices | cut -f1
0000
0001
0002
0010
0020
0030
PAGE 305
0038
0048
0049
004a
0060
0078
0080
0090
0098
00a0
$ tree /sys/bus/pci/devices/
/sys/bus/pci/devices/
|-- 0000:00:00.0 -> ../../../devices/pci0000:00/0000:00:00.0
|-- 0000:00:00.1 -> ../../../devices/pci0000:00/0000:00:00.1
|-- 0000:00:00.2 -> ../../../devices/pci0000:00/0000:00:00.2
|-- 0000:00:02.0 -> ../../../devices/pci0000:00/0000:00:02.0
|-- 0000:00:04.0 -> ../../../devices/pci0000:00/0000:00:04.0
|-- 0000:00:06.0 -> ../../../devices/pci0000:00/0000:00:06.0
|-- 0000:00:07.0 -> ../../../devices/pci0000:00/0000:00:07.0
|-- 0000:00:09.0 -> ../../../devices/pci0000:00/0000:00:09.0
|-- 0000:00:09.1 -> ../../../devices/pci0000:00/0000:00:09.1
|-- 0000:00:09.2 -> ../../../devices/pci0000:00/0000:00:09.2
|-- 0000:00:0c.0 -> ../../../devices/pci0000:00/0000:00:0c.0
|-- 0000:00:0f.0 -> ../../../devices/pci0000:00/0000:00:0f.0
|-- 0000:00:10.0 -> ../../../devices/pci0000:00/0000:00:10.0
|-- 0000:00:12.0 -> ../../../devices/pci0000:00/0000:00:12.0
|-- 0000:00:13.0 -> ../../../devices/pci0000:00/0000:00:13.0
`-- 0000:00:14.0 -> ../../../devices/pci0000:00/0000:00:14.0
All three lists of devices are sorted in the same order, since lspci uses the /proc files as
its source of information. Taking the VGA video controller as an example, 0x00a0
means 0000:00:14.0 when split into domain (16 bits), bus (8 bits), device (5 bits) and
function (3 bits).
The hardware circuitry of each peripheral board answers queries pertaining to three
address spaces: memory locations, I/O ports, and configuration registers. The first
two address spaces are shared by all the devices on the same PCI bus (i.e., when you
access a memory location, all the devices on that PCI bus see the bus cycle at the
same time). The configuration space, on the other hand, exploits geographical
addressing. Configuration queries address only one slot at a time, so they never collide.
As far as the driver is concerned, memory and I/O regions are accessed in the usual
ways via inb, readb, and so forth. Configuration transactions, on the other hand, are
performed by calling specific kernel functions to access configuration registers. With
regard to interrupts, every PCI slot has four interrupt pins, and each device function
can use one of them without being concerned about how those pins are routed to the
CPU. Such routing is the responsibility of the computer platform and is implemented
outside of the PCI bus. Since the PCI specification requires interrupt lines to
be shareable, even a processor with a limited number of IRQ lines, such as the x86,
can host many PCI interface boards (each with four interrupt pins).
PAGE 306
The I/O space in a PCI bus uses a 32-bit address bus (leading to 4 GB of I/O ports),
while the memory space can be accessed with either 32-bit or 64-bit addresses. 64-bit
addresses are available on more recent platforms. Addresses are supposed to be
unique to one device, but software may erroneously configure two devices to the
same address, making it impossible to access either one. But this problem never
occurs unless a driver is willingly playing with registers it shouldn't touch. The good
news is that every memory and I/O address region offered by the interface board can
be remapped by means of configuration transactions. That is, the firmware initializes
PCI hardware at system boot, mapping each region to a different address to
avoid collisions.* The addresses to which these regions are currently mapped can be
read from the configuration space, so the Linux driver can access its devices without
probing. After reading the configuration registers, the driver can safely access its
hardware.
The PCI configuration space consists of 256 bytes for each device function (except
for PCI Express devices, which have 4 KB of configuration space for each function),
and the layout of the configuration registers is standardized. Four bytes of the configuration
space hold a unique function ID, so the driver can identify its device by looking
for the specific ID for that peripheral. In summary, each device board is
geographically addressed to retrieve its configuration registers; the information in
those registers can then be used to perform normal I/O access, without the need for
further geographic addressing.
It should be clear from this description that the main innovation of the PCI interface
standard over ISA is the configuration address space. Therefore, in addition to the
usual driver code, a PCI driver needs the ability to access the configuration space, in
order to save itself from risky probing tasks.
For the remainder of this chapter, we use the word device to refer to a device function,
because each function in a multifunction board acts as an independent entity.
When we refer to a device, we mean the tuple "domain number, bus number, device
number, and function number."
Boot Time
To see how PCI works, we start from system boot, since that's when the devices are
configured.
* Actually, that configuration is not restricted to the time the system boots; hotpluggable devices, for example,
cannot be available at boot time and appear later instead. The main point here is that the device driver must
not change the address of I/O or memory regions.
You'll find the ID of any device in its own hardware manual. A list is included in the file pci.ids, part
of thepciutils package and the kernel sources; it doesn't pretend to be complete but just lists the most renowned
vendors and devices. The kernel version of this file will not be included in future kernel series.
PAGE 307
When power is applied to a PCI device, the hardware remains inactive. In other
words, the device responds only to configuration transactions. At power on, the
device has no memory and no I/O ports mapped in the computer's address space;
every other device-specific feature, such as interrupt reporting, is disabled as well.
Fortunately, every PCI motherboard is equipped with PCI-aware firmware, called the
BIOS, NVRAM, or PROM, depending on the platform. The firmware offers access to
the device configuration address space by reading and writing registers in the PCI
controller.
At system boot, the firmware (or the Linux kernel, if so configured) performs configuration
transactions with every PCI peripheral in order to allocate a safe place for
each address region it offers. By the time a device driver accesses the device, its memory
and I/O regions have already been mapped into the processor's address space.
The driver can change this default assignment, but it never needs to do that.
As suggested, the user can look at the PCI device list and the devices' configuration
registers by reading /proc/bus/pci/devices and /proc/bus/pci/*/*. The former is a text file
with (hexadecimal) device information, and the latter are binary files that report a
snapshot of the configuration registers of each device, one file per device. The individual
PCI device directories in the sysfs tree can be found in /sys/bus/pci/devices. A
PCI device directory contains a number of different files:
$ tree /sys/bus/pci/devices/0000:00:10.0
/sys/bus/pci/devices/0000:00:10.0
|-- class
|-- config
|-- detach_state
|-- device
|-- irq
|-- power
| `-- state
|-- resource
|-- subsystem_device
|-- subsystem_vendor
`-- vendor
The file config is a binary file that allows the raw PCI config information to be read
from the device (just like the /proc/bus/pci/*/* provides.) The files vendor, device,
subsystem_device, subsystem_vendor, and class all refer to the specific values of this
PCI device (all PCI devices provide this information.) The file irq shows the current
IRQ assigned to this PCI device, and the file resource shows the current memory
resources allocated by this device.
PAGE 308
Configuration Registers and Initialization
In this section, we look at the configuration registers that PCI devices contain. All
PCI devices feature at least a 256-byte address space. The first 64 bytes are standardized,
while the rest are device dependent. Figure 12-2 shows the layout of the device-independent
configuration space.
Figure 12-2. The standardized PCI configuration registers
As the figure shows, some of the PCI configuration registers are required and some
are optional. Every PCI device must contain meaningful values in the required registers,
whereas the contents of the optional registers depend on the actual capabilities
of the peripheral. The optional fields are not used unless the contents of the required
fields indicate that they are valid. Thus, the required fields assert the board's capabilities,
including whether the other fields are usable.
It's interesting to note that the PCI registers are always little-endian. Although the
standard is designed to be architecture independent, the PCI designers sometimes
show a slight bias toward the PC environment. The driver writer should be careful
about byte ordering when accessing multibyte configuration registers; code that
works on the PC might not work on other platforms. The Linux developers have
taken care of the byte-ordering problem (see the next section, "Accessing the Configuration
Space"), but the issue must be kept in mind. If you ever need to convert data
from host order to PCI order or vice versa, you can resort to the functions defined in
<asm/byteorder.h>, introduced in Chapter 11, knowing that PCI byte order is little-endian.
PAGE 309
Describing all the configuration items is beyond the scope of this book. Usually, the
technical documentation released with each device describes the supported registers.
What we're interested in is how a driver can look for its device and how it can access
the device's configuration space.
Three or five PCI registers identify a device: vendorID, deviceID, and class are the
three that are always used. Every PCI manufacturer assigns proper values to these
read-only registers, and the driver can use them to look for the device. Additionally,
the fields subsystem vendorID and subsystem deviceID are sometimes set by the vendor
to further differentiate similar devices.
Let's look at these registers in more detail:
vendorID
This 16-bit register identifies a hardware manufacturer. For instance, every Intel
device is marked with the same vendor number, 0x8086. There is a global registry
of such numbers, maintained by the PCI Special Interest Group, and manufacturers
must apply to have a unique number assigned to them.
deviceID
This is another 16-bit register, selected by the manufacturer; no official registration
is required for the device ID. This ID is usually paired with the vendor ID to
make a unique 32-bit identifier for a hardware device. We use the word signature
to refer to the vendor and device ID pair. A device driver usually relies on
the signature to identify its device; you can find what value to look for in the
hardware manual for the target device.
class
Every peripheral device belongs to a class. The class register is a 16-bit value
whose top 8 bits identify the "base class" (or group). For example, "ethernet"
and "token ring" are two classes belonging to the "network" group, while the
"serial" and "parallel" classes belong to the "communication" group. Some drivers
can support several similar devices, each of them featuring a different signature
but all belonging to the same class; these drivers can rely on the class
register to identify their peripherals, as shown later.
subsystem vendorID
subsystem deviceID
These fields can be used for further identification of a device. If the chip is a
generic interface chip to a local (onboard) bus, it is often used in several completely
different roles, and the driver must identify the actual device it is talking
with. The subsystem identifiers are used to this end.
Using these different identifiers, a PCI driver can tell the kernel what kind of devices it
supports. The struct pci_device_id structure is used to define a list of the different
PAGE 310
types of PCI devices that a driver supports. This structure contains the following
fields:
__u32 vendor;
__u32 device;
These specify the PCI vendor and device IDs of a device. If a driver can handle
any vendor or device ID, the value PCI_ANY_ID should be used for these fields.
__u32 subvendor;
__u32 subdevice;
These specify the PCI subsystem vendor and subsystem device IDs of a device. If
a driver can handle any type of subsystem ID, the value PCI_ANY_ID should be
used for these fields.
__u32 class;
__u32 class_mask;
These two values allow the driver to specify that it supports a type of PCI class
device. The different classes of PCI devices (a VGA controller is one example)
are described in the PCI specification. If a driver can handle any type of subsystem
ID, the value PCI_ANY_ID should be used for these fields.
kernel_ulong_t driver_data;
This value is not used to match a device but is used to hold information that the
PCI driver can use to differentiate between different devices if it wants to.
There are two helper macros that should be used to initialize a struct pci_device_id
structure:
PCI_DEVICE(vendor, device)
This creates a struct pci_device_id that matches only the specific vendor and
device ID. The macro sets the subvendor and subdevice fields of the structure to
PCI_ANY_ID.
PCI_DEVICE_CLASS(device_class, device_class_mask)
This creates a struct pci_device_id that matches a specific PCI class.
An example of using these macros to define the type of devices a driver supports can
be found in the following kernel files:
drivers/usb/host/ehci-hcd.c:
static const struct pci_device_id pci_ids[ ] = { {
/* handle any USB 2.0 EHCI controller */
PCI_DEVICE_CLASS(((PCI_CLASS_SERIAL_USB << 8) | 0x20), ~0),
.driver_data = (unsigned long) &ehci_driver,
},
{ /* end: all zeroes */ }
};
drivers/i2c/busses/i2c-i810.c:
PAGE 311
static struct pci_device_id i810_ids[ ] = {
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82810_IG1) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82810_IG3) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82810E_IG) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82815_CGC) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82845G_IG) },
{ 0, },
};
These examples create a list of struct pci_device_id structures, with an empty structure
set to all zeros as the last value in the list. This array of IDs is used in the struct pci_driver
(described below), and it is also used to tell user space which devices this specific driver supports.
MODULE_DEVICE_TABLE
This pci_device_id structure needs to be exported to user space to allow the hotplug
and module loading systems know what module works with what hardware devices.
The macro MODULE_DEVICE_TABLE accomplishes this. An example is:
MODULE_DEVICE_TABLE(pci, i810_ids);
This statement creates a local variable called __mod_pci_device_table that points to
the list of struct pci_device_id. Later in the kernel build process, the depmod program
searches all modules for the symbol __mod_pci_device_table. If that symbol is
found, it pulls the data out of the module and adds it to the file /lib/modules/
KERNEL_VERSION/modules.pcimap. After depmod completes, all PCI devices that
are supported by modules in the kernel are listed, along with their module names, in
that file. When the kernel tells the hotplug system that a new PCI device has been
found, the hotplug system uses the modules.pcimap file to find the proper driver to
load.
Registering a PCI Driver
The main structure that all PCI drivers must create in order to be registered with the
kernel properly is the struct pci_driver structure. This structure consists of a number
of function callbacks and variables that describe the PCI driver to the PCI core.
Here are the fields in this structure that a PCI driver needs to be aware of:
const char *name;
The name of the driver. It must be unique among all PCI drivers in the kernel
and is normally set to the same name as the module name of the driver. It shows
up in sysfs under /sys/bus/pci/drivers/ when the driver is in the kernel.
const struct pci_device_id *id_table;
Pointer to the struct pci_device_id table described earlier in this chapter.
PAGE 312
int (*probe) (struct pci_dev *dev, const struct pci_device_id *id);
Pointer to the probe function in the PCI driver. This function is called by the PCI
core when it has a struct pci_dev that it thinks this driver wants to control. A
pointer to the struct pci_device_id that the PCI core used to make this decision
is also passed to this function. If the PCI driver claims the struct pci_dev that is
passed to it, it should initialize the device properly and return 0. If the driver
does not want to claim the device, or an error occurs, it should return a negative
error value. More details about this function follow later in this chapter.
void (*remove) (struct pci_dev *dev);
Pointer to the function that the PCI core calls when the struct pci_dev is being
removed from the system, or when the PCI driver is being unloaded from the
kernel. More details about this function follow later in this chapter.
int (*suspend) (struct pci_dev *dev, u32 state);
Pointer to the function that the PCI core calls when the struct pci_dev is being
suspended. The suspend state is passed in the state variable. This function is
optional; a driver does not have to provide it.
int (*resume) (struct pci_dev *dev);
Pointer to the function that the PCI core calls when the struct pci_dev is being
resumed. It is always called after suspend has been called. This function is
optional; a driver does not have to provide it.
In summary, to create a proper struct pci_driver structure, only four fields need to
be initialized:
static struct pci_driver pci_driver = {
.name = "pci_skel",
.id_table = ids,
.probe = probe,
.remove = remove,
};
To register the struct pci_driver with the PCI core, a call to pci_register_driver is
made with a pointer to the struct pci_driver. This is traditionally done in the module
initialization code for the PCI driver:
static int __init pci_skel_init(void)
{
return pci_register_driver(&pci_driver);
}
Note that the pci_register_driver function either returns a negative error number or 0
if everything was registered successfully. It does not return the number of devices
that were bound to the driver or an error number if no devices were bound to the
PAGE 313
driver. This is a change from kernels prior to the 2.6 release and was done because of
the following situations:
- On systems that support PCI hotplug, or CardBus systems, a PCI device can
appear or disappear at any point in time. It is helpful if drivers can be loaded
before the device appears, to reduce the time it takes to initialize a device.
- The 2.6 kernel allows new PCI IDs to be dynamically allocated to a driver after it
has been loaded. This is done through the file new_id that is created in all PCI
driver directories in sysfs. This is very useful if a new device is being used that
the kernel doesn't know about just yet. A user can write the PCI ID values to the
new_id file, and then the driver binds to the new device. If a driver was not
allowed to load until a device was present in the system, this interface would not
be able to work.
When the PCI driver is to be unloaded, the struct pci_driver needs to be unregistered
from the kernel. This is done with a call to pci_unregister_driver. When this call
happens, any PCI devices that were currently bound to this driver are removed, and
the remove function for this PCI driver is called before the pci_unregister_driver function
returns.
static void __exit pci_skel_exit(void)
{
pci_unregister_driver(&pci_driver);
}
Old-Style PCI Probing
In older kernel versions, the function, pci_register_driver, was not always used by
PCI drivers. Instead, they would either walk the list of PCI devices in the system by
hand, or they would call a function that could search for a specific PCI device. The
ability to walk the list of PCI devices in the system within a driver has been removed
from the 2.6 kernel in order to prevent drivers from crashing the kernel if they happened
to modify the PCI device lists while a device was being removed at the same
time.
If the ability to find a specific PCI device is really needed, the following functions are
available:
struct pci_dev *pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from);
This function scans the list of PCI devices currently present in the system, and if
the input arguments match the specified vendor and device IDs, it increments
the reference count on the struct pci_dev variable found, and returns it to the
caller. This prevents the structure from disappearing without any notice and
ensures that the kernel does not oops. After the driver is done with the struct
pci_dev returned by the function, it must call the function pci_dev_put to decrement
PAGE 314
the usage count properly back to allow the kernel to clean up the device if
it is removed.
The
from argument is used to get hold of multiple devices with the same signature;
the argument should point to the last device that has been found, so that
the search can continue instead of restarting from the head of the list. To find
the first device,
from is specified as
NULL. If no (further) device is found,
NULL is
returned.
An example of how to use this function properly is:
struct pci_dev *dev;
dev = pci_get_device(PCI_VENDOR_FOO, PCI_DEVICE_FOO, NULL);
if (dev) {
/* Use the PCI device */
...
pci_dev_put(dev);
}
This function can not be called from interrupt context. If it is, a warning is
printed out to the system log.
struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device,
unsigned int ss_vendor, unsigned int ss_device, struct pci_dev *from);
This function works just like pci_get_device, but it allows the subsystem vendor
and subsystem device IDs to be specified when looking for the device.
This function can not be called from interrupt context. If it is, a warning is
printed out to the system log.
struct pci_dev *pci_get_slot(struct pci_bus *bus, unsigned int devfn);
This function searches the list of PCI devices in the system on the specified
struct pci_bus for the specified device and function number of the PCI device. If
a device is found that matches, its reference count is incremented and a pointer
to it is returned. When the caller is finished accessing the struct pci_dev, it must
call pci_dev_put.
All of these functions can not be called from interrupt context. If they are, a warning
is printed out to the system log.
Enabling the PCI Device
In the probe function for the PCI driver, before the driver can access any device resource
(I/O region or interrupt) of the PCI device, the driver must call the pci_enable_device
function:
int pci_enable_device(struct pci_dev *dev);
This function actually enables the device. It wakes up the device and in some
cases also assigns its interrupt line and I/O regions. This happens, for example,
with CardBus devices (which have been made completely equivalent to PCI at
the driver level).
PAGE 315
Accessing the Configuration Space
After the driver has detected the device, it usually needs to read from or write to the
three address spaces: memory, port, and configuration. In particular, accessing the
configuration space is vital to the driver, because it is the only way it can find out
where the device is mapped in memory and in the I/O space.
Because the microprocessor has no way to access the configuration space directly,
the computer vendor has to provide a way to do it. To access configuration space,
the CPU must write and read registers in the PCI controller, but the exact implementation
is vendor dependent and not relevant to this discussion, because Linux offers a
standard interface to access the configuration space.
As far as the driver is concerned, the configuration space can be accessed through 8bit,
16-bit, or 32-bit data transfers. The relevant functions are prototyped in <linux/
pci.h>:
int pci_read_config_byte(struct pci_dev *dev, int where, u8 *val);
int pci_read_config_word(struct pci_dev *dev, int where, u16 *val);
int pci_read_config_dword(struct pci_dev *dev, int where, u32 *val);
Read one, two, or four bytes from the configuration space of the device identified
by dev. The where argument is the byte offset from the beginning of the configuration
space. The value fetched from the configuration space is returned
through the val pointer, and the return value of the functions is an error code.
The word and dword functions convert the value just read from little-endian to
the native byte order of the processor, so you need not deal with byte ordering.
int pci_write_config_byte(struct pci_dev *dev, int where, u8 val);
int pci_write_config_word(struct pci_dev *dev, int where, u16 val);
int pci_write_config_dword(struct pci_dev *dev, int where, u32 val);
Write one, two, or four bytes to the configuration space. The device is identified
by dev as usual, and the value being written is passed as val. The word and
dword functions convert the value to little-endian before writing to the peripheral
device.
All of the previous functions are implemented as inline functions that really call the
following functions. Feel free to use these functions instead of the above in case the
driver does not have access to a struct pci_dev at any paticular moment in time:
int pci_bus_read_config_byte (struct pci_bus *bus, unsigned int devfn, int where, u8 *val);
int pci_bus_read_config_word (struct pci_bus *bus, unsigned int devfn, int where, u16 *val);
int pci_bus_read_config_dword (struct pci_bus *bus, unsigned int devfn, int where, u32 *val);
Just like the pci_read_ functions, but struct pci_bus * and devfn variables are
needed instead of a struct pci_dev *.
PAGE 316
int pci_bus_write_config_byte (struct pci_bus *bus, unsigned int devfn, int where, u8 val);
int pci_bus_write_config_word (struct pci_bus *bus, unsigned int devfn, int where, u16 val);
int pci_bus_write_config_dword (struct pci_bus *bus, unsigned int devfn, int where, u32 val);
Just like the pci_write_ functions, but struct pci_bus * and devfn variables are
needed instead of a struct pci_dev *.
The best way to address the configuration variables using the pci_read_ functions is
by means of the symbolic names defined in <linux/pci.h>. For example, the following
small function retrieves the revision ID of a device by passing the symbolic name
for where to pci_read_config_byte:
static unsigned char skel_get_revision(struct pci_dev *dev)
{
u8 revision;
pci_read_config_byte(dev, PCI_REVISION_ID, &revision);
return revision;
}
Accessing the I/O and Memory Spaces
A PCI device implements up to six I/O address regions. Each region consists of either
memory or I/O locations. Most devices implement their I/O registers in memory
regions, because it's generally a saner approach (as explained in the section "I/O
Ports and I/O Memory," in Chapter 9). However, unlike normal memory, I/O registers
should not be cached by the CPU because each access can have side effects. The
PCI device that implements I/O registers as a memory region marks the difference by
setting a "memory-is-prefetchable" bit in its configuration register.* If the memory
region is marked as prefetchable, the CPU can cache its contents and do all sorts of
optimization with it; nonprefetchable memory access, on the other hand, can't be
optimized because each access can have side effects, just as with I/O ports. Peripherals
that map their control registers to a memory address range declare that range as
nonprefetchable, whereas something like video memory on PCI boards is prefetchable.
In this section, we use the word region to refer to a generic I/O address space
that is memory-mapped or port-mapped.
An interface board reports the size and current location of its regions using configuration
registers--the six 32-bit registers shown in Figure 12-2, whose symbolic names
are PCI_BASE_ADDRESS_0 through PCI_BASE_ADDRESS_5. Since the I/O space defined by
PCI is a 32-bit address space, it makes sense to use the same configuration interface
* The information lives in one of the low-order bits of the base address PCI registers.
The bits are defined in <linux/pci.h>.
PAGE 317
for memory and I/O. If the device uses a 64-bit address bus, it can declare regions in
the 64-bit memory space by using two consecutive PCI_BASE_ADDRESS registers for each
region, low bits first. It is possible for one device to offer both 32-bit regions and 64bit
regions.
In the kernel, the I/O regions of PCI devices have been integrated into the generic
resource management. For this reason, you don't need to access the configuration
variables in order to know where your device is mapped in memory or I/O space.
The preferred interface for getting region information consists of the following
functions:
unsigned long pci_resource_start(struct pci_dev *dev, int bar);
The function returns the first address (memory address or I/O port number)
associated with one of the six PCI I/O regions. The region is selected by the integer
bar (the base address register), ranging from 0-5 (inclusive).
unsigned long pci_resource_end(struct pci_dev *dev, int bar);
The function returns the last address that is part of the I/O region number bar.
Note that this is the last usable address, not the first address after the region.
unsigned long pci_resource_flags(struct pci_dev *dev, int bar);
This function returns the flags associated with this resource.
Resource flags are used to define some features of the individual resource. For PCI
resources associated with PCI I/O regions, the information is extracted from the base
address registers, but can come from elsewhere for resources not associated with PCI
devices.
All resource flags are defined in <linux/ioport.h>; the most important are:
IORESOURCE_IO
IORESOURCE_MEM
If the associated I/O region exists, one and only one of these flags is set.
IORESOURCE_PREFETCH
IORESOURCE_READONLY
These flags tell whether a memory region is prefetchable and/or write protected.
The latter flag is never set for PCI resources.
By making use of the pci_resource_ functions, a device driver can completely ignore
the underlying PCI registers, since the system already used them to structure
resource information.
PCI Interrupts
As far as interrupts are concerned, PCI is easy to handle. By the time Linux boots,
the computer's firmware has already assigned a unique interrupt number to the
device, and the driver just needs to use it. The interrupt number is stored in configuration
register 60 (PCI_INTERRUPT_LINE), which is one byte wide. This allows for as
PAGE 318
many as 256 interrupt lines, but the actual limit depends on the CPU being used.
The driver doesn't need to bother checking the interrupt number, because the value
found in PCI_INTERRUPT_LINE is guaranteed to be the right one.
If the device doesn't support interrupts, register 61 (PCI_INTERRUPT_PIN) is 0; otherwise,
it's nonzero. However, since the driver knows if its device is interrupt driven or
not, it doesn't usually need to read PCI_INTERRUPT_PIN.
Thus, PCI-specific code for dealing with interrupts just needs to read the configuration
byte to obtain the interrupt number that is saved in a local variable, as shown in
the following code. Beyond that, the information in Chapter 10 applies.
result = pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &myirq);
if (result) {
/* deal with error */
}
The rest of this section provides additional information for the curious reader but
isn't needed for writing drivers.
A PCI connector has four interrupt pins, and peripheral boards can use any or all of
them. Each pin is individually routed to the motherboard's interrupt controller, so
interrupts can be shared without any electrical problems. The interrupt controller is
then responsible for mapping the interrupt wires (pins) to the processor's hardware;
this platform-dependent operation is left to the controller in order to achieve platform
independence in the bus itself.
The read-only configuration register located at PCI_INTERRUPT_PIN is used to tell the
computer which single pin is actually used. It's worth remembering that each device
board can host up to eight devices; each device uses a single interrupt pin and reports
it in its own configuration register. Different devices on the same device board can
use different interrupt pins or share the same one.
The PCI_INTERRUPT_LINE register, on the other hand, is read/write. When the computer
is booted, the firmware scans its PCI devices and sets the register for each
device according to how the interrupt pin is routed for its PCI slot. The value is
assigned by the firmware, because only the firmware knows how the motherboard
routes the different interrupt pins to the processor. For the device driver, however,
the PCI_INTERRUPT_LINE register is read-only. Interestingly, recent versions of the
Linux kernel under some circumstances can assign interrupt lines without resorting
to the BIOS.
Hardware Abstractions
We complete the discussion of PCI by taking a quick look at how the system handles
the plethora of PCI controllers available on the marketplace. This is just an
informational section, meant to show the curious reader how the object-oriented layout
of the kernel extends down to the lowest levels.
PAGE 319
The mechanism used to implement hardware abstraction is the usual structure containing
methods. It's a powerful technique that adds just the minimal overhead of
dereferencing a pointer to the normal overhead of a function call. In the case of PCI
management, the only hardware-dependent operations are the ones that read and
write configuration registers, because everything else in the PCI world is accomplished
by directly reading and writing the I/O and memory address spaces, and
those are under direct control of the CPU.
Thus, the relevant structure for configuration register access includes only two fields:
struct pci_ops {
int (*read)(struct pci_bus *bus, unsigned int devfn, int where, int size,
u32 *val);
int (*write)(struct pci_bus *bus, unsigned int devfn, int where, int size,
u32 val);
};
The structure is defined in <linux/pci.h> and used by drivers/pci/pci.c, where the
actual public functions are defined.
The two functions that act on the PCI configuration space have more overhead
than dereferencing a pointer; they use cascading pointers due to the high object-orientedness
of the code, but the overhead is not an issue in operations that are
performed quite rarely and never in speed-critical paths. The actual implementation
of pci_read_config_byte(dev, where, val), for instance, expands to:
dev->bus->ops->read(bus, devfn, where, 8, val);
The various PCI buses in the system are detected at system boot, and that's when the
struct pci_bus items are created and associated with their features, including the ops
field.
Implementing hardware abstraction via "hardware operations" data structures is typical
in the Linux kernel. One important example is the struct alpha_machine_vector
data structure. It is defined in <asm-alpha/machvec.h> and takes care of everything
that may change across different Alpha-based computers.
A Look Back: ISA
The ISA bus is quite old in design and is a notoriously poor performer, but it still
holds a good part of the market for extension devices. If speed is not important and
you want to support old motherboards, an ISA implementation is preferable to PCI.
An additional advantage of this old standard is that if you are an electronic hobbyist,
you can easily build your own ISA devices, something definitely not possible with
PCI.
PAGE 320
On the other hand, a great disadvantage of ISA is that it's tightly bound to the PC
architecture; the interface bus has all the limitations of the 80286 processor and
causes endless pain to system programmers. The other great problem with the ISA
design (inherited from the original IBM PC) is the lack of geographical addressing,
which has led to many problems and lengthy unplug-rejumper-plug-test cycles to
add new devices. It's interesting to note that even the oldest Apple II computers were
already exploiting geographical addressing, and they featured jumperless expansion
boards.
Despite its great disadvantages, ISA is still used in several unexpected places. For
example, the VR41xx series of MIPS processors used in several palmtops features an
ISA-compatible expansion bus, strange as it seems. The reason behind these unexpected
uses of ISA is the extreme low cost of some legacy hardware, such as 8390based
Ethernet cards, so a CPU with ISA electrical signaling can easily exploit the
awful, but cheap, PC devices.
Hardware Resources
An ISA device can be equipped with I/O ports, memory areas, and interrupt lines.
Even though the x86 processors support 64 KB of I/O port memory (i.e., the processor
asserts 16 address lines), some old PC hardware decodes only the lowest 10
address lines. This limits the usable address space to 1024 ports, because any address
in the range 1 KB to 64 KB is mistaken for a low address by any device that decodes
only the low address lines. Some peripherals circumvent this limitation by mapping
only one port into the low kilobyte and using the high address lines to select between
different device registers. For example, a device mapped at 0x340 can safely use port
0x740, 0xB40, and so on.
If the availability of I/O ports is limited, memory access is still worse. An ISA device
can use only the memory range between 640 KB and 1 MB and between 15 MB and
16 MB for I/O register and device control. The 640-KB to 1-MB range is used by the
PC BIOS, by VGA-compatible video boards, and by various other devices, leaving little
space available for new devices. Memory at 15 MB, on the other hand, is not
directly supported by Linux, and hacking the kernel to support it is a waste of programming
time nowadays.
The third resource available to ISA device boards is interrupt lines. A limited number
of interrupt lines is routed to the ISA bus, and they are shared by all the interface
boards. As a result, if devices aren't properly configured, they can find themselves
using the same interrupt lines.
PAGE 321
Although the original ISA specification doesn't allow interrupt sharing across
devices, most device boards allow it.* Interrupt sharing at the software level is
described in the section "Interrupt Sharing," in Chapter 10.
ISA Programming
As far as programming is concerned, there's no specific aid in the kernel or the BIOS
to ease access to ISA devices (like there is, for example, for PCI). The only facilities
you can use are the registries of I/O ports and IRQ lines, described in the section
"Installing an Interrupt Handler" in Chapter 10.
The programming techniques shown throughout the first part of this book apply to
ISA devices; the driver can probe for I/O ports, and the interrupt line must be auto-detected
with one of the techniques shown in the section "Auto-Detecting the IRQ Number" in Chapter 10.
The helper functions isa_readb and friends have been briefly introduced in the section
"Using I/O Memory" in Chapter 9, and there's nothing more to say about them.
The Plug-and-Play Specification
Some new ISA device boards follow peculiar design rules and require a special initialization
sequence intended to simplify installation and configuration of add-on interface
boards. The specification for the design of these boards is called plug and play
(PnP) and consists of a cumbersome rule set for building and configuring jumperless
ISA devices. PnP devices implement relocatable I/O regions; the PC's BIOS is responsible
for the relocation--reminiscent of PCI.
In short, the goal of PnP is to obtain the same flexibility found in PCI devices without
changing the underlying electrical interface (the ISA bus). To this end, the specs
define a set of device-independent configuration registers and a way to geographically
address the interface boards, even though the physical bus doesn't carry perboard
(geographical) wiring--every ISA signal line connects to every available slot.
Geographical addressing works by assigning a small integer, called the card select
number (CSN), to each PnP peripheral in the computer. Each PnP device features a
unique serial identifier, 64 bits wide, that is hardwired into the peripheral board.
CSN assignment uses the unique serial number to identify the PnP devices. But the
CSNs can be assigned safely only at boot time, which requires the BIOS to be PnP
* The problem with interrupt sharing is a matter of electrical engineering: if a device
drives the signal line inactive--by applying a low-impedance voltage level--the interrupt
can't be shared. If, on the other hand, the device uses a pull-up resistor to the inactive
logic level, sharing is possible. This is the norm nowadays. However, there's still a
potential risk of losing interrupt events since ISA interrupts are edge triggered instead of
level triggered. Edge-triggered interrupts are easier to implement in hardware but don't
lend themselves to safe sharing.
PAGE 322
aware. For this reason, old computers require the user to obtain and insert a specific
configuration diskette, even if the device is PnP capable.
Interface boards following the PnP specs are complicated at the hardware level. They
are much more elaborate than PCI boards and require complex software. It's not
unusual to have difficulty installing these devices, and even if the installation goes
well, you still face the performance constraints and the limited I/O space of the ISA
bus. It's much better to install PCI devices whenever possible and enjoy the new
technology instead.
If you are interested in the PnP configuration software, you can browse drivers/net/
3c509.c, whose probing function deals with PnP devices. The 2.6 kernel saw a lot of
work in the PnP device support area, so a lot of the inflexible interfaces have been
cleaned up compared to previous kernel releases.
PC/104 and PC/104+
Currently in the industrial world, two bus architectures are quite fashionable: PC/104
and PC/104+. Both are standard in PC-class single-board computers.
Both standards refer to specific form factors for printed circuit boards, as well as
electrical/mechanical specifications for board interconnections. The practical advantage
of these buses is that they allow circuit boards to be stacked vertically using a
plug-and-socket kind of connector on one side of the device.
The electrical and logical layout of the two buses is identical to ISA (PC/104) and
PCI (PC/104+), so software won't notice any difference between the usual desktop
buses and these two.
Other PC Buses
PCI and ISA are the most commonly used peripheral interfaces in the PC world, but
they aren't the only ones. Here's a summary of the features of other buses found in
the PC market.
MCA
Micro Channel Architecture (MCA) is an IBM standard used in PS/2 computers and
some laptops. At the hardware level, Micro Channel has more features than ISA. It
supports multimaster DMA, 32-bit address and data lines, shared interrupt lines, and
geographical addressing to access per-board configuration registers. Such registers
are called Programmable Option Select (POS), but they don't have all the features of
the PCI registers. Linux support for Micro Channel includes functions that are
exported to modules.
PAGE 323
A device driver can read the integer value MCA_bus to see if it is running on a Micro
Channel computer. If the symbol is a preprocessor macro, the macro MCA_bus__is_a_macro
is defined as well. If MCA_bus__is_a_macro is undefined, then MCA_bus is an integer
variable exported to modularized code. Both MCA_BUS and MCA_bus__is_a_macro
are defined in <asm/processor.h>.
EISA
The Extended ISA (EISA) bus is a 32-bit extension to ISA, with a compatible interface
connector; ISA device boards can be plugged into an EISA connector. The additional
wires are routed under the ISA contacts.
Like PCI and MCA, the EISA bus is designed to host jumperless devices, and it has
the same features as MCA: 32-bit address and data lines, multimaster DMA, and
shared interrupt lines. EISA devices are configured by software, but they don't need
any particular operating system support. EISA drivers already exist in the Linux kernel
for Ethernet devices and SCSI controllers.
An EISA driver checks the value EISA_bus to determine if the host computer carries
an EISA bus. Like MCA_bus, EISA_bus is either a macro or a variable, depending on
whether EISA_bus__is_a_macro is defined. Both symbols are defined in <asm/
processor.h>.
The kernel has full EISA support for devices with sysfs and resource management
functionality. This is located in the drivers/eisa directory.
VLB
Another extension to ISA is the VESA Local Bus (VLB) interface bus, which extends
the ISA connectors by adding a third lengthwise slot. A device can just plug into this
extra connector (without plugging in the two associated ISA connectors), because
the VLB slot duplicates all important signals from the ISA connectors. Such "stand alone"
VLB peripherals not using the ISA slot are rare, because most devices need to
reach the back panel so that their external connectors are available.
The VESA bus is much more limited in its capabilities than the EISA, MCA, and PCI
buses and is disappearing from the market. No special kernel support exists for VLB.
However, both the Lance Ethernet driver and the IDE disk driver in Linux 2.0 can
deal with VLB versions of their devices.
SBus
While most computers nowadays are equipped with a PCI or ISA interface bus, most
older SPARC-based workstations use SBus to connect their peripherals.
PAGE 324
SBus is quite an advanced design, although it has been around for a long time. It is
meant to be processor independent (even though only SPARC computers use it) and
is optimized for I/O peripheral boards. In other words, you can't plug additional
RAM into SBus slots (RAM expansion boards have long been forgotten even in the
ISA world, and PCI does not support them either). This optimization is meant to
simplify the design of both hardware devices and system software, at the expense of
some additional complexity in the motherboard.
This I/O bias of the bus results in peripherals using virtual addresses to transfer data,
thus bypassing the need to allocate a contiguous DMA buffer. The motherboard is
responsible for decoding the virtual addresses and mapping them to physical
addresses. This requires attaching an MMU (memory management unit) to the bus;
the chipset in charge of the task is called IOMMU. Although somehow more complex
than using physical addresses on the interface bus, this design is greatly simplified
by the fact that SPARC processors have always been designed by keeping the
MMU core separate from the CPU core (either physically or at least conceptually).
Actually, this design choice is shared by other smart processor designs and is beneficial
overall. Another feature of this bus is that device boards exploit massive geographical
addressing, so there's no need to implement an address decoder in every
peripheral or to deal with address conflicts.
SBus peripherals use the Forth language in their PROMs to initialize themselves.
Forth was chosen because the interpreter is lightweight and, therefore, can be easily
implemented in the firmware of any computer system. In addition, the SBus specification
outlines the boot process, so that compliant I/O devices fit easily into the system
and are recognized at system boot. This was a great step to support multiplatform
devices; it's a completely different world from the PC-centric ISA stuff we
were used to. However, it didn't succeed for a variety of commercial reasons.
Although current kernel versions offer quite full-featured support for SBus devices,
the bus is used so little nowadays that it's not worth covering in detail here. Interested
readers can look at source files in arch/sparc/kernel and arch/sparc/mm.
NuBus
Another interesting, but nearly forgotten, interface bus is NuBus. It is found on older
Mac computers (those with the M68k family of CPUs).
All of the bus is memory-mapped (like everything with the M68k), and the devices
are only geographically addressed. This is good and typical of Apple, as the much
PAGE 325
older Apple II already had a similar bus layout. What is bad is that it's almost impossible
to find documentation on NuBus, due to the close-everything policy Apple has
always followed with its Mac computers (and unlike the previous Apple II, whose
source code and schematics were available at little cost).
The file drivers/nubus/nubus.c includes almost everything we know about this bus,
and it's interesting reading; it shows how much hard reverse engineering developers
had to do.
External Buses
One of the most recent entries in the field of interface buses is the whole class of
external buses. This includes USB, FireWire, and IEEE1284 (parallel-port-based
external bus). These interfaces are somewhat similar to older and not-so-external
technology, such as PCMCIA/CardBus and even SCSI.
Conceptually, these buses are neither full-featured interface buses (like PCI is) nor
dumb communication channels (like the serial ports are). It's hard to classify the
software that is needed to exploit their features, as it's usually split into two levels:
the driver for the hardware controller (like drivers for PCI SCSI adaptors or PCI controllers
introduced in the section "The PCI Interface") and the driver for the specific
"client" device (like sd.c handles generic SCSI disks and so-called PCI drivers deal
with cards plugged in the bus).
Quick Reference
This section summarizes the symbols introduced in the chapter:
#include <linux/pci.h>
Header that includes symbolic names for the PCI registers and several vendor
and device ID values.
struct pci_dev;
Structure that represents a PCI device within the kernel.
struct pci_driver;
Structure that represents a PCI driver. All PCI drivers must define this.
struct pci_device_id;
Structure that describes the types of PCI devices this driver supports.
int pci_register_driver(struct pci_driver *drv);
int pci_module_init(struct pci_driver *drv);
void pci_unregister_driver(struct pci_driver *drv);
Functions that register or unregister a PCI driver from the kernel.
PAGE 326
struct pci_dev *pci_find_device(unsigned int vendor, unsigned int device, struct pci_dev *from);
struct pci_dev *pci_find_device_reverse(unsigned int vendor, unsigned int device, const struct pci_dev *from);
struct pci_dev *pci_find_subsys (unsigned int vendor, unsigned int device, unsigned int ss_vendor, unsigned int ss_device, const struct pci_dev *from);
struct pci_dev *pci_find_class(unsigned int class, struct pci_dev *from);
Functions that search the device list for devices with a specific signature or those
belonging to a specific class. The return value is NULL if none is found. from is
used to continue a search; it must be NULL the first time you call either function,
and it must point to the device just found if you are searching for more devices.
These functions are not recommended to be used, use the pci_get_ variants
instead.
struct pci_dev *pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from);
struct pci_dev *pci_get_subsys(unsigned int vendor, unsigned int device, unsigned int ss_vendor, unsigned int ss_device, struct pci_dev *from);
struct pci_dev *pci_get_slot(struct pci_bus *bus, unsigned int devfn);
Functions that search the device list for devices with a specific signature or
belonging to a specific class. The return value is NULL if none is found. from is
used to continue a search; it must be NULL the first time you call either function,
and it must point to the device just found if you are searching for more devices.
The structure returned has its reference count incremented, and after the caller is
finished with it, the function pci_dev_put must be called.
int pci_read_config_byte(struct pci_dev *dev, int where, u8 *val);
int pci_read_config_word(struct pci_dev *dev, int where, u16 *val);
int pci_read_config_dword(struct pci_dev *dev, int where, u32 *val);
int pci_write_config_byte (struct pci_dev *dev, int where, u8 *val);
int pci_write_config_word (struct pci_dev *dev, int where, u16 *val);
int pci_write_config_dword (struct pci_dev *dev, int where, u32 *val);
Functions that read or write a PCI configuration register. Although the Linux
kernel takes care of byte ordering, the programmer must be careful about byte
ordering when assembling multibyte values from individual bytes. The PCI bus
is little-endian.
int pci_enable_device(struct pci_dev *dev);
Enables a PCI device.
unsigned long pci_resource_start(struct pci_dev *dev, int bar);
unsigned long pci_resource_end(struct pci_dev *dev, int bar);
unsigned long pci_resource_flags(struct pci_dev *dev, int bar);
Functions that handle PCI device resources.
PAGE 327
CHAPTER 13
USB Drivers
The universal serial bus (USB) is a connection between a host computer and a number
of peripheral devices. It was originally created to replace a wide range of slow
and different buses--the parallel, serial, and keyboard connections--with a single
bus type that all devices could connect to.* USB has grown beyond these slow connections
and now supports almost every type of device that can be connected to a
PC. The latest revision of the USB specification added high-speed connections with a
theoretical speed limit of 480 MBps.
Topologically, a USB subsystem is not laid out as a bus; it is rather a tree built out of
several point-to-point links. The links are four-wire cables (ground, power, and two
signal wires) that connect a device and a hub, just like twisted-pair Ethernet. The
USB host controller is in charge of asking every USB device if it has any data to send.
Because of this topology, a USB device can never start sending data without first
being asked to by the host controller. This configuration allows for a very easy plugand-play
type of system, whereby devices can be automatically configured by the
host computer.
The bus is very simple at the technological level, as it's a single-master implementation
in which the host computer polls the various peripheral devices. Despite this
intrinsic limitation, the bus has some interesting features, such as the ability for a
device to request a fixed bandwidth for its data transfers in order to reliably support
video and audio I/O. Another important feature of USB is that it acts merely as a
communication channel between the device and the host, without requiring specific
meaning or structure to the data it delivers.
* Portions of this chapter are based on the in-kernel documentation for the Linux kernel USB code, which were
written by the kernel USB developers and released under the GPL.
Actually, some structure is there, but it mostly reduces to a requirement for the communication to fit into
one of a few predefined classes: a keyboard won't allocate bandwidth, for example, while some video cameras
will.
PAGE 328
The USB protocol specifications define a set of standards that any device of a specific
type can follow. If a device follows that standard, then a special driver for that
device is not necessary. These different types are called classes and consist of things
like storage devices, keyboards, mice, joysticks, network devices, and modems.
Other types of devices that do not fit into these classes require a special vendor-specific
driver to be written for that specific device. Video devices and USB-to-serial
devices are a good example where there is no defined standard, and a driver is
needed for every different device from different manufacturers.
These features, together with the inherent hotplug capability of the design, make
USB a handy, low-cost mechanism to connect (and disconnect) several devices to the
computer without the need to shut the system down, open the cover, and swear over
screws and wires.
The Linux kernel supports two main types of USB drivers: drivers on a host system
and drivers on a device. The USB drivers for a host system control the USB devices
that are plugged into it, from the host's point of view (a common USB host is a desktop
computer.) The USB drivers in a device, control how that single device looks to
the host computer as a USB device. As the term "USB device drivers" is very confusing,
the USB developers have created the term "USB gadget drivers" to describe the
drivers that control a USB device that connects to a computer (remember that Linux
also runs in those tiny embedded devices, too.) This chapter details how the USB system
that runs on a desktop computer works. USB gadget drivers are outside the
realm of this book at this point in time.
As Figure 13-1 shows, USB drivers live between the different kernel subsytems
(block, net, char, etc.) and the USB hardware controllers. The USB core provides an
interface for USB drivers to use to access and control the USB hardware, without
having to worry about the different types of USB hardware controllers that are
present on the system.
USB Device Basics
A USB device is a very complex thing, as described in the official USB documentation
(available at http://www.usb.org). Fortunately, the Linux kernel provides a subsystem
called the USB core to handle most of the complexity. This chapter describes
the interaction between a driver and the USB core. Figure 13-2 shows how USB
devices consist of configurations, interfaces, and endpoints and how USB drivers
bind to USB interfaces, not the entire USB device.
Endpoints
The most basic form of USB communication is through something called an endpoint.
A USB endpoint can carry data in only one direction, either from the host
PAGE 329
Figure 13-1. USB driver overview
Figure 13-2. USB device overview
computer to the device (called an OUT endpoint) or from the device to the host computer
(called an IN endpoint). Endpoints can be thought of as unidirectional pipes.
A USB endpoint can be one of four different types that describe how the data is
transmitted:
CONTROL
Control endpoints are used to allow access to different parts of the USB device.
They are commonly used for configuring the device, retrieving information
about the device, sending commands to the device, or retrieving status reports
about the device. These endpoints are usually small in size. Every USB device has
PAGE 330
a control endpoint called "endpoint 0" that is used by the USB core to configure
the device at insertion time. These transfers are guaranteed by the USB protocol
to always have enough reserved bandwidth to make it through to the device.
INTERRUPT
Interrupt endpoints transfer small amounts of data at a fixed rate every time the
USB host asks the device for data. These endpoints are the primary transport
method for USB keyboards and mice. They are also commonly used to send data
to USB devices to control the device, but are not generally used to transfer large
amounts of data. These transfers are guaranteed by the USB protocol to always
have enough reserved bandwidth to make it through.
BULK
Bulk endpoints transfer large amounts of data. These endpoints are usually
much larger (they can hold more characters at once) than interrupt endpoints.
They are common for devices that need to transfer any data that must get
through with no data loss. These transfers are not guaranteed by the USB protocol
to always make it through in a specific amount of time. If there is not enough
room on the bus to send the whole BULK packet, it is split up across multiple
transfers to or from the device. These endpoints are common on printers, storage,
and network devices.
ISOCHRONOUS
Isochronous endpoints also transfer large amounts of data, but the data is not
always guaranteed to make it through. These endpoints are used in devices that
can handle loss of data, and rely more on keepinga constant stream of data
flowing. Real-time data collections, such as audio and video devices, almost
always use these endpoints.
Control and bulk endpoints are used for asynchronous data transfers, whenever the
driver decides to use them. Interrupt and isochronous endpoints are periodic. This
means that these endpoints are set up to transfer data at fixed times continuously,
which causes their bandwidth to be reserved by the USB core.
USB endpoints are described in the kernel with the structure struct usb_host_endpoint.
This structure contains the real endpoint information in another structure called
struct usb_endpoint_descriptor. The latter structure contains all of the USB-specific
data in the exact format that the device itself specified. The fields of this structure that
drivers care about are:
bEndpointAddress
This is the USB address of this specific endpoint. Also included in this 8-bit
value is the direction of the endpoint. The bitmasks USB_DIR_OUT and USB_DIR_IN
can be placed against this field to determine if the data for this endpoint is
directed to the device or to the host.
bmAttributes
This is the type of endpoint. The bitmask USB_ENDPOINT_XFERTYPE_MASK should
be placed against this value in order to determine if the endpoint is of type
PAGE 331
USB_ENDPOINT_XFER_ISOC, USB_ENDPOINT_XFER_BULK, or of
type USB_ENDPOINT_XFER_INT. These macros define a isochronous, bulk,
and interrupt endpoint, respectively.
wMaxPacketSize
This is the maximum size in bytes that this endpoint can handle at once. Note
that it is possible for a driver to send amounts of data to an endpoint that is bigger
than this value, but the data will be divided up into wMaxPacketSize chunks
when actually transmitted to the device. For high-speed devices, this field can be
used to support a high-bandwidth mode for the endpoint by using a few extra
bits in the upper part of the value. See the USB specification for more details
about how this is done.
bInterval
If this endpoint is of type interrupt, this value is the interval setting for the endpoint--that
is, the time between interrupt requests for the endpoint. The value is
represented in milliseconds.
The fields of this structure do not have a "traditional" Linux kernel naming scheme.
This is because these fields directly correspond to the field names in the USB specification.
The USB kernel programmers felt that it was more important to use the specified
names, so as to reduce confusion when reading the specification, than it was to
have variable names that look familiar to Linux programmers.
Interfaces
USB endpoints are bundled up into interfaces. USB interfaces handle only one type of
a USB logical connection, such as a mouse, a keyboard, or a audio stream. Some USB
devices have multiple interfaces, such as a USB speaker that might consist of two
interfaces: a USB keyboard for the buttons and a USB audio stream. Because a USB
interface represents basic functionality, each USB driver controls an interface; so, for
the speaker example, Linux needs two different drivers for one hardware device.
USB interfaces may have alternate settings, which are different choices for parameters
of the interface. The initial state of a interface is in the first setting, numbered 0.
Alternate settings can be used to control individual endpoints in different ways, such
as to reserve different amounts of USB bandwidth for the device. Each device with an
isochronous endpoint uses alternate settings for the same interface.
USB interfaces are described in the kernel with the struct usb_interface structure.
This structure is what the USB core passes to USB drivers and is what the USB driver
then is in charge of controlling. The important fields in this structure are:
struct usb_host_interface *altsetting
An array of interface structures containing all of the alternate settings that may
be selected for this interface. Each struct usb_host_interface consists of a set of
PAGE 332
endpoint configurations as defined by the struct usb_host_endpoint structure
described above. Note that these interface structures are in no particular order.
unsigned num_altsetting
The number of alternate settings pointed to by the altsetting pointer.
struct usb_host_interface *cur_altsetting
A pointer into the array altsetting, denoting the currently active setting for this
interface.
int minor
If the USB driver bound to this interface uses the USB major number, this variable
contains the minor number assigned by the USB core to the interface. This
is valid only after a successful call to usb_register_dev (described later in this
chapter).
There are other fields in the struct usb_interface structure, but USB drivers do not
need to be aware of them.
Configurations
USB interfaces are themselves bundled up into configurations. A USB device can have
multiple configurations and might switch between them in order to change the state
of the device. For example, some devices that allow firmware to be downloaded to
them contain multiple configurations to accomplish this. A single configuration can
be enabled only at one point in time. Linux does not handle multiple configuration
USB devices very well, but, thankfully, they are rare.
Linux describes USB configurations with the structure struct usb_host_config and
entire USB devices with the structure struct usb_device. USB device drivers do not
generally ever need to read or write to any values in these structures, so they are not
defined in detail here. The curious reader can find descriptions of them in the file
include/linux/usb.h in the kernel source tree.
A USB device driver commonly has to convert data from a given struct usb_interface
structure into a struct usb_device structure that the USB core needs for a wide range of
function calls. To do this, the function interface_to_usbdev is provided. Hopefully, in the
future, all USB calls that currently need a struct usb_device will be converted to take a
struct usb_interface parameter and will not require the drivers to do the conversion.
So to summarize, USB devices are quite complex and are made up of lots of different
logical units. The relationships among these units can be simply described as follows:
- Devices usually have one or more configurations.
- Configurations often have one or more interfaces.
- Interfaces usually have one or more settings.
- Interfaces have zero or more endpoints.
PAGE 333
USB and Sysfs
Due to the complexity of a single USB physical device, the representation of that
device in sysfs is also quite complex. Both the physical USB device (as represented by
a struct usb_device) and the individual USB interfaces (as represented by a struct
usb_interface) are shown in sysfs as individual devices. (This is because both of
those structures contain a struct device structure.) As an example, for a simple USB
mouse that contains only one USB interface, the following would be the sysfs directory
tree for that device:
/sys/devices/pci0000:00/0000:00:09.0/usb2/2-1
|-- 2-1:1.0
| |-- bAlternateSetting
| |-- bInterfaceClass
| |-- bInterfaceNumber
| |-- bInterfaceProtocol
| |-- bInterfaceSubClass
| |-- bNumEndpoints
| |-- detach_state
| |-- iInterface
| `-- power
| `-- state
|-- bConfigurationValue
|-- bDeviceClass
|-- bDeviceProtocol
|-- bDeviceSubClass
|-- bMaxPower
|-- bNumConfigurations
|-- bNumInterfaces
|-- bcdDevice
|-- bmAttributes
|-- detach_state
|-- devnum
|-- idProduct
|-- idVendor
|-- maxchild
|-- power
| `-- state
|-- speed
`-- version
The struct usb_device is represented in the tree at:
/sys/devices/pci0000:00/0000:00:09.0/usb2/2-1
while the USB interface for the mouse--the interface that the USB mouse driver is
bound to--is located at the directory:
/sys/devices/pci0000:00/0000:00:09.0/usb2/2-1/2-1:1.0
To help understand what this longdevice path means, we describe how the kernel
labels the USB devices.
PAGE 334
The first USB device is a root hub. This is the USB controller, usually contained in a
PCI device. The controller is so named because it controls the whole USB bus connected
to it. The controller is a bridge between the PCI bus and the USB bus, as well
as being the first USB device on that bus.
All root hubs are assigned a unique number by the USB core. In our example, the
root hub is called usb2, as it is the second root hub that was registered with the USB
core. There is no limit on the number of root hubs that can be contained in a single
system at any time.
Every device that is on a USB bus takes the number of the root hub as the first number
in its name. That is followed by a - character and then the number of the port
that the device is plugged into. As the device in our example is plugged into the first
port, a 1 is added to the name. So the device name for the main USB mouse device is
2-1. Because this USB device contains one interface, that causes another device in the
tree to be added to the sysfs path. The naming scheme for USB interfaces is the
device name up to this point: in our example, it's 2-1 followed by a colon and the
USB configuration number, then a period and the interface number. So for this
example, the device name is 2-1:1.0 because it is the first configuration and has
interface number zero.
So to summarize, the USB sysfs device naming scheme is:
root_hub-hub_port:config.interface
As the devices go further down in the USB tree, and as more and more USB hubs are
used, the hub port number is added to the string following the previous hub port
number in the chain. For a two-deep tree, the device name looks like:
root_hub-hub_port-hub_port:config.interface
As can be seen in the previous directory listing of the USB device and interface, all of
the USB specific information is available directly through sysfs (for example, the idVendor,
idProduct, and bMaxPower information). One of these files, bConfigurationValue,
can be written to in order to change the active USB configuration that is being used.
This is useful for devices that have multiple configurations, when the kernel is unable
to determine what configuration to select in order to properly operate the device. A
number of USB modems need to have the proper configuration value written to this file
in order to have the correct USB driver bind to the device.
Sysfs does not expose all of the different parts of a USB device, as it stops at the interface
level. Any alternate configurations that the device may contain are not shown, as
well as the details of the endpoints associated with the interfaces. This information
can be found in the usbfs filesystem, which is mounted in the /proc/bus/usb/ directory
on the system. The file /proc/bus/usb/devices does show all of the same information
exposed in sysfs, as well as the alternate configuration and endpoint information
PAGE 335
for all USB devices that are present in the system. usbfs also allows user-space programs
to directly talk to USB devices, which has enabled a lot of kernel drivers to be
moved out to user space, where it is easier to maintain and debug. The USB scanner
driver is a good example of this, as it is no longer present in the kernel because its
functionality is now contained in the user-space SANE library programs.
USB Urbs
The USB code in the Linux kernel communicates with all USB devices using something called
a urb (USB request block). This request block is described with the
struct urb structure and can be found in the include/linux/usb.h file.
A urb is used to send or receive data to or from a specific USB endpoint on a specific
USB device in an asynchronous manner. It is used much like a kiocb structure is used
in the filesystem async I/O code or as a struct skbuff is used in the networking code.
A USB device driver may allocate many urbs for a single endpoint or may reuse a single
urb for many different endpoints, depending on the need of the driver. Every endpoint
in a device can handle a queue of urbs, so that multiple urbs can be sent to the
same endpoint before the queue is empty. The typical lifecycle of a urb is as follows:
- Created by a USB device driver.
- Assigned to a specific endpoint of a specific USB device.
- Submitted to the USB core, by the USB device driver.
- Submitted to the specific USB host controller driver for the specified device by the USB core.
- Processed by the USB host controller driver that makes a USB transfer to the device.
- When the urb is completed, the USB host controller driver notifies the USB device driver.
Urbs can also be canceled any time by the driver that submitted the urb, or by the
USB core if the device is removed from the system. urbs are dynamically created and
contain an internal reference count that enables them to be automatically freed when
the last user of the urb releases it.
The procedure described in this chapter for handling urbs is useful, because it permits
streaming and other complex, overlapping communications that allow drivers
to achieve the highest possible data transfer speeds. But less cumbersome procedures
are available if you just want to send individual bulk or control messages and
do not care about data throughput rates. (See the section "USB Transfers Without Urbs.")
PAGE 336
struct urb
The fields of the struct urb structure that matter to a USB device driver are:
struct usb_device *dev
Pointer to the struct usb_device to which this urb is sent. This variable must be
initialized by the USB driver before the urb can be sent to the USB core.
unsigned int pipe
Endpoint information for the specific
struct usb_device that this urb is to be
sent to. This variable must be initialized by the USB driver before the urb can be sent to the USB core.
To set fields of this structure, the driver uses the following functions as appropriate,
depending on the direction of traffic. Note that every endpoint can be of only one type.
unsigned int usb_sndctrlpipe(struct usb_device *dev, unsigned int endpoint)
Specifies a control OUT endpoint for the specified USB device with the specified endpoint number.
unsigned int usb_rcvctrlpipe(struct usb_device *dev, unsigned int endpoint)
Specifies a control IN endpoint for the specified USB device with the specified endpoint number.
unsigned int usb_sndbulkpipe(struct usb_device *dev, unsigned int endpoint)
Specifies a bulk OUT endpoint for the specified USB device with the specified endpoint number.
unsigned int usb_rcvbulkpipe(struct usb_device *dev, unsigned int endpoint)
Specifies a bulk IN endpoint for the specified USB device with the specified endpoint number.
unsigned int usb_sndintpipe(struct usb_device *dev, unsigned int endpoint)
Specifies an interrupt OUT endpoint for the specified USB device with the specified endpoint number.
unsigned int usb_rcvintpipe(struct usb_device *dev, unsigned int endpoint)
Specifies an interrupt IN endpoint for the specified USB device with the specified endpoint number.
PAGE 337
unsigned int usb_sndisocpipe(struct usb_device *dev, unsigned int endpoint)
Specifies an isochronous OUT endpoint for the specified USB device with
the specified endpoint number.
unsigned int usb_rcvisocpipe(struct usb_device *dev, unsigned int endpoint)
Specifies an isochronous IN endpoint for the specified USB device with the
specified endpoint number.
unsigned int transfer_flags
This variable can be set to a number of different bit values, depending on what
the USB driver wants to happen to the urb. The available values are:
URB_SHORT_NOT_OK
When set, it specifies that any short read on an IN endpoint that might
occur should be treated as an error by the USB core. This value is useful only
for urbs that are to be read from the USB device, not for write urbs.
URB_ISO_ASAP
If the urb is isochronous, this bit can be set if the driver wants the urb to be
scheduled, as soon as the bandwidth utilization allows it to be, and to set
the start_frame variable in the urb at that point. If this bit is not set for an
isochronous urb, the driver must specify the start_frame value and must be
able to recover properly if the transfer cannot start at that moment. See the
upcoming section about isochronous urbs for more information.
URB_NO_TRANSFER_DMA_MAP
Should be set when the urb contains a DMA buffer to be transferred. The
USB core uses the buffer pointed to by the transfer_dma variable and not the
buffer pointed to by the transfer_buffer variable.
URB_NO_SETUP_DMA_MAP
Like the URB_NO_TRANSFER_DMA_MAP bit, this bit is used for control urbs that
have a DMA buffer already set up. If it is set, the USB core uses the buffer
pointed to by the setup_dma variable instead of the setup_packet variable.
URB_ASYNC_UNLINK
If set, the call to usb_unlink_urb for this urb returns almost immediately,
and the urb is unlinked in the background. Otherwise, the function waits
until the urb is completely unlinked and finished before returning. Use this
bit with care, because it can make synchronization issues very difficult to
debug.
PAGE 338
URB_NO_FSBR
Used by only the UHCI USB Host controller driver and tells it to not try to
do Front Side Bus Reclamation logic. This bit should generally not be set,
because machines with a UHCI host controller create a lot of CPU overhead,
and the PCI bus is saturated waiting on a urb that sets this bit.
URB_ZERO_PACKET
If set, a bulk out urb finishes by sending a short packet containing no data
when the data is aligned to an endpoint packet boundary. This is needed by
some broken USB devices (such as a number of USB to IR devices) in order
to work properly.
URB_NO_INTERRUPT
If set, the hardware may not generate an interrupt when the urb is finished.
This bit should be used with care and only when queuing multiple urbs to
the same endpoint. The USB core functions use this in order to do DMA
buffer transfers.
void *transfer_buffer
Pointer to the buffer to be used when sending data to the device (for an OUT
urb) or when receiving data from the device (for an IN urb). In order for the host
controller to properly access this buffer, it must be created with a call to kmalloc,
not on the stack or statically. For control endpoints, this buffer is for the data
stage of the transfer.
dma_addr_t transfer_dma
Buffer to be used to transfer data to the USB device using DMA.
int transfer_buffer_length
The length of the buffer pointed to by the transfer_buffer or the transfer_dma
variable (as only one can be used for a urb). If this is 0, neither transfer buffers
are used by the USB core.
For an OUT endpoint, if the endpoint maximum size is smaller than the value
specified in this variable, the transfer to the USB device is broken up into smaller
chunks in order to properly transfer the data. This large transfer occurs in consecutive
USB frames. It is much faster to submit a large block of data in one urb,
and have the USB host controller split it up into smaller pieces, than it is to send
smaller buffers in consecutive order.
unsigned char *setup_packet
Pointer to the setup packet for a control urb. It is transferred before the data in
the transfer buffer. This variable is valid only for control urbs.
dma_addr_t setup_dma
DMA buffer for the setup packet for a control urb. It is transferred before the
data in the normal transfer buffer. This variable is valid only for control urbs.
PAGE 339
usb_complete_t complete
Pointer to the completion handler function that is called by the USB core when
the urb is completely transferred or when an error occurs to the urb. Within this
function, the USB driver may inspect the urb, free it, or resubmit it for another
transfer. (See the section "CompletingUrbs: The Completion Callback Handler"
for more details about the completion handler.)
The
usb_complete_t typedef is defined as:
typedef void (*usb_complete_t)(struct urb *, struct pt_regs *);
void *context
Pointer to a data blob that can be set by the USB driver. It can be used in the
completion handler when the urb is returned to the driver. See the following section
for more details about this variable.
int actual_length
When the urb is finished, this variable is set to the actual length of the data
either sent by the urb (for OUT urbs) or received by the urb (for IN urbs.) For
IN urbs, this must be used instead of the transfer_buffer_length variable,
because the data received could be smaller than the whole buffer size.
int status
When the urb is finished, or being processed by the USB core, this variable is set
to the current status of the urb. The only time a USB driver can safely access this
variable is in the urb completion handler function (described in the section
"CompletingUrbs: The Completion Callback Handler"). This restriction is to
prevent race conditions that occur while the urb is being processed by the USB
core. For isochronous urbs, a successful value (0) in this variable merely indicates
whether the urb has been unlinked. To obtain a detailed status on isochronous
urbs, the
iso_frame_desc variables should be checked.
Valid values for this variable include:
0
The urb transfer was successful.
-ENOENT
The urb was stopped by a call to usb_kill_urb.
-ECONNRESET
The urb was unlinked by a call to usb_unlink_urb, and the transfer_flags
variable of the urb was set to URB_ASYNC_UNLINK.
-EINPROGRESS
The urb is still being processed by the USB host controllers. If your driver
ever sees this value, it is a bug in your driver.
-EPROTO
One of the following errors occurred with this urb:
- A bitstuff error happened during the transfer.
- No response packet was received in time by the hardware.
PAGE 340
-EILSEQ
There was a CRC mismatch in the urb transfer.
-EPIPE
The endpoint is now stalled. If the endpoint involved is not a control endpoint,
this error can be cleared through a call to the function usb_clear_halt.
-ECOMM
Data was received faster during the transfer than it could be written to system
memory. This error value happens only for an IN urb.
-ENOSR
Data could not be retrieved from the system memory during the transfer fast
enough to keep up with the requested USB data rate. This error value happens
only for an OUT urb.
-EOVERFLOW
A "babble" error happened to the urb. A "babble" error occurs when the
endpoint receives more data than the endpoint's specified maximum packet
size.
-EREMOTEIO
Occurs only if the URB_SHORT_NOT_OK flag is set in the urb's transfer_flags
variable and means that the full amount of data requested by the urb was
not received.
-ENODEV
The USB device is now gone from the system.
-EXDEV
Occurs only for a isochronous urb and means that the transfer was only partially
completed. In order to determine what was transferred, the driver
must look at the individual frame status.
-EINVAL
Something very bad happened with the urb. The USB kernel documentation
describes what this value means:
ISO madness, if this happens: Log off and go home
It also can happen if a parameter is incorrectly set in the urb stucture or if an
incorrect function parameter in the
usb_submit_urb call submitted the urb to
the USB core.
-ESHUTDOWN
There was a severe error with the USB host controller driver; it has now
been disabled, or the device was disconnected from the system, and the urb
was submitted after the device was removed. It can also occur if the configuration
was changed for the device, while the urb was submitted to the
device.
PAGE 341
Generally, the error values -EPROTO, -EILSEQ, and -EOVERFLOW indicate hardware
problems with the device, the device firmware, or the cable connecting the device to the computer.
int start_frame
Sets or returns the initial frame number for isochronous transfers to use.
int interval
The interval at which the urb is polled. This is valid only for interrupt or isochronous
urbs. The value's units differ depending on the speed of the device. For
low-speed and full-speed devices, the units are frames, which are equivalent to
milliseconds. For devices, the units are in microframes, which is equivalent to
units of 1/8 milliseconds. This value must be set by the USB driver for isochronous
or interrupt urbs before the urb is sent to the USB core.
int number_of_packets
Valid only for isochronous urbs and specifies the number of isochronous transfer
buffers to be handled by this urb. This value must be set by the USB driver
for isochronous urbs before the urb is sent to the USB core.
int error_count
Set by the USB core only for isochronous urbs after their completion. It specifies
the number of isochronous transfers that reported any type of error.
struct usb_iso_packet_descriptor iso_frame_desc[0]
Valid only for isochronous urbs. This variable is an array of the
struct usb_iso_packet_descriptor
structures that make up this urb. This structure allows a single
urb to define a number of isochronous transfers at once. It is also used to collect
the transfer status of each individual transfer.
The
struct usb_iso_packet_descriptor is made up of the following fields:
unsigned int offset
The offset into the transfer buffer (starting at 0 for the first byte) where this
packet's data is located.
unsigned int length
The length of the transfer buffer for this packet.
unsigned int actual_length
The length of the data received into the transfer buffer for this isochronous
packet.
unsigned int status
The status of the individual isochronous transfer of this packet. It can take
the same return values as the main struct urb structure's status variable.
Creating and Destroying Urbs
The struct urb structure must never be created statically in a driver or within
another structure, because that would break the reference counting scheme used by
PAGE 342
the USB core for urbs. It must be created with a call to the usb_alloc_urb function.
This function has the prototype:
struct urb *usb_alloc_urb(int iso_packets, int mem_flags);
The first parameter, iso_packets, is the number of isochronous packets this urb
should contain. If you do not want to create an isochronous urb, this variable should
be set to 0. The second parameter, mem_flags, is the same type of flag that is passed
to the kmalloc function call to allocate memory from the kernel (see the section "The
Flags Argument" in Chapter 8 for the details on these flags). If the function is successful
in allocating enough space for the urb, a pointer to the urb is returned to the
caller. If the return value is NULL, some error occurred within the USB core, and the
driver needs to clean up properly.
After a urb has been created, it must be properly initialized before it can be used by
the USB core. See the next sections for how to initialize different types of urbs.
In order to tell the USB core that the driver is finished with the urb, the driver must
call the usb_free_urb function. This function only has one argument:
void usb_free_urb(struct urb *urb);
The argument is a pointer to the struct urb you want to release. After this function
is called, the urb structure is gone, and the driver cannot access it any more.
Interrupt urbs
The function usb_fill_int_urb is a helper function to properly initialize a urb to be
sent to a interrupt endpoint of a USB device:
void usb_fill_int_urb(struct urb *urb, struct usb_device *dev,
unsigned int pipe, void *transfer_buffer,
int buffer_length, usb_complete_t complete,
void *context, int interval);
This function contains a lot of parameters:
struct urb *urb
A pointer to the urb to be initialized.
struct usb_device *dev
The USB device to which this urb is to be sent.
unsigned int pipe
The specific endpoint of the USB device to which this urb is to be sent. This
value is created with the previously mentioned usb_sndintpipe or usb_rcvintpipe
functions.
void *transfer_buffer
A pointer to the buffer from which outgoing data is taken or into which incoming data
is received. Note that this can not be a static buffer and must be created
with a call to kmalloc.
PAGE 343
int buffer_length
The length of the buffer pointed to by the transfer_buffer pointer.
usb_complete_t complete
Pointer to the completion handler that is called when this urb is completed.
void *context
Pointer to the blob that is added to the urb structure for later retrieval by the
completion handler function.
int interval
The interval at which that this urb should be scheduled. See the previous
description of the struct urb structure to find the proper units for this value.
Bulk urbs
Bulk urbs are initialized much like interrupt urbs. The function that does this is
usb_fill_bulk_urb, and it looks like:
void usb_fill_bulk_urb(struct urb *urb, struct usb_device *dev,
unsigned int pipe, void *transfer_buffer,
int buffer_length, usb_complete_t complete,
void *context);
The function parameters are all the same as in the usb_fill_int_urb function. However,
there is no interval parameter because bulk urbs have no interval value. Please
note that the unsigned int pipe variable must be initialized with a call to the usb_sndbulkpipe
or usb_rcvbulkpipe function.
The usb_fill_int_urb function does not set the transfer_flags variable in the urb, so
any modification to this field has to be done by the driver itself.
Control urbs
Control urbs are initialized almost the same way as bulk urbs, with a call to the function
usb_fill_control_urb:
void usb_fill_control_urb(struct urb *urb, struct usb_device *dev,
unsigned int pipe, unsigned char *setup_packet,
void *transfer_buffer, int buffer_length,
usb_complete_t complete, void *context);
The function parameters are all the same as in the usb_fill_bulk_urb function, except
that there is a new parameter, unsigned char *setup_packet, which must point to the
setup packet data that is to be sent to the endpoint. Also, the unsigned int pipe variable
must be initialized with a call to the usb_sndctrlpipe or usb_rcvictrlpipe function.
The usb_fill_control_urb function does not set the transfer_flags variable in the urb,
so any modification to this field has to be done by the driver itself. Most drivers do
not use this function, as it is much simpler to use the synchronous API calls as
described in the section "USB Transfers Without Urbs."
PAGE 344
Isochronous urbs
Isochronous urbs unfortunately do not have an initializer function like the interrupt,
control, and bulk urbs do. So they must be initialized "by hand" in the driver before
they can be submitted to the USB core. The following is an example of how to properly
initialize this type of urb. It was taken from the konicawc.c kernel driver located
in the drivers/usb/media directory in the main kernel source tree.
urb->dev = dev;
urb->context = uvd;
urb->pipe = usb_rcvisocpipe(dev, uvd->video_endp-1);
urb->interval = 1;
urb->transfer_flags = URB_ISO_ASAP;
urb->transfer_buffer = cam->sts_buf[i];
urb->complete = konicawc_isoc_irq;
urb->number_of_packets = FRAMES_PER_DESC;
urb->transfer_buffer_length = FRAMES_PER_DESC;
for (j=0; j < FRAMES_PER_DESC; j++) {
urb->iso_frame_desc[j].offset = j;
urb->iso_frame_desc[j].length = 1;
}
Submitting Urbs
Once the urb has been properly created and initialized by the USB driver, it is ready
to be submitted to the USB core to be sent out to the USB device. This is done with a
call to the function usb_submit_urb:
int usb_submit_urb(struct urb *urb, int mem_flags);
The urb parameter is a pointer to the urb that is to be sent to the device. The mem_flags
parameter is equivalent to the same parameter that is passed to the kmalloc call and is
used to tell the USB core how to allocate any memory buffers at this moment in time.
After a urb has been submitted to the USB core successfully, it should never try to
access any fields of the urb structure until the complete function is called.
Because the function usb_submit_urb can be called at any time (including from
within an interrupt context), the specification of the mem_flags variable must be correct.
There are really only three valid values that should be used, depending on when
usb_submit_urb is being called:
GFP_ATOMIC
This value should be used whenever the following are true:
- The caller is within a urb completion handler, an interrupt, a bottom half, a tasklet, or a timer callback.
- The caller is holding a spinlock or rwlock. Note that if a semaphore is being held, this value is not necessary.
- The current->state is not TASK_RUNNING. The state is
always TASK_RUNNING unless the driver has changed the current state itself.
PAGE 345
GFP_NOIO
This value should be used if the driver is in the block I/O patch. It should also be
used in the error handling path of all storage-type devices.
GFP_KERNEL
This should be used for all other situations that do not fall into one of the previously
mentioned categories.
Completing Urbs: The Completion Callback Handler
If the call to usb_submit_urb was successful, transferring control of the urb to the
USB core, the function returns 0; otherwise, a negative error number is returned. If
the function succeeds, the completion handler of the urb (as specified by the complete
function pointer) is called exactly once when the urb is completed. When this
function is called, the USB core is finished with the URB, and control of it is now
returned to the device driver.
There are only three ways a urb can be finished and have the complete function
called:
- The urb is successfully sent to the device, and the device returns the proper
acknowledgment. For an OUT urb, the data was successfully sent, and for an IN
urb, the requested data was successfully received. If this has happened, the
status variable in the urb is set to 0.
- Some kind of error happened when sending or receiving data from the device.
This is noted by the error value in the status variable in the urb structure.
- The urb was "unlinked" from the USB core. This happens either when the driver
tells the USB core to cancel a submitted urb with a call to usb_unlink_urb or
usb_kill_urb, or when a device is removed from the system and a urb had been
submitted to it.
An example of how to test for the different return values within a urb completion call
is shown later in this chapter.
Canceling Urbs
To stop a urb that has been submitted to the USB core, the functions usb_kill_urb or
usb_unlink_urb should be called:
int usb_kill_urb(struct urb *urb);
int usb_unlink_urb(struct urb *urb);
The urb parameter for both of these functions is a pointer to the urb that is to be
canceled.
PAGE 346
When the function is usb_kill_urb, the urb lifecycle is stopped. This function is
usually used when the device is disconnected from the system, in the disconnect
callback.
For some drivers, the usb_unlink_urb function should be used to tell the USB core to
stop an urb. This function does not wait for the urb to be fully stopped before
returning to the caller. This is useful for stopping the urb while in an interrupt handler
or when a spinlock is held, as waiting for a urb to fully stop requires the ability
for the USB core to put the calling process to sleep. This function requires
that the URB_ASYNC_UNLINK flag value be set in the urb that is being asked to be
stopped in order to work properly.
Writing a USB Driver
The approach to writing a USB device driver is similar to a pci_driver: the driver registers
its driver object with the USB subsystem and later uses vendor and device identifiers
to tell if its hardware has been installed.
What Devices Does the Driver Support?
The struct usb_device_id structure provides a list of different types of USB devices
that this driver supports. This list is used by the USB core to decide which driver to
give a device to, and by the hotplug scripts to decide which driver to automatically
load when a specific device is plugged into the system.
The struct usb_device_id structure is defined with the following fields:
__u16 match_flags
Determines which of the following fields in the structure the device should be
matched against. This is a bit field defined by the different USB_DEVICE_ID_MATCH_*
values specified in the include/linux/mod_devicetable.h file. This field is usually
never set directly but is initialized by the USB_DEVICE type macros described later.
__u16 idVendor
The USB vendor ID for the device. This number is assigned by the USB forum to
its members and cannot be made up by anyone else.
__u16 idProduct
The USB product ID for the device. All vendors that have a vendor ID assigned
to them can manage their product IDs however they choose to.
__u16 bcdDevice_lo
__u16 bcdDevice_hi
Define the low and high ends of the range of the vendor-assigned product version
number. The bcdDevice_hi value is inclusive; its value is the number of the
highest-numbered device. Both of these values are expressed in binary-coded
PAGE 347
decimal (BCD) form. These variables, combined with the idVendor and
idProduct, are used to define a specific version of a device.
__u8 bDeviceClass
__u8 bDeviceSubClass
__u8 bDeviceProtocol
Define the class, subclass, and protocol of the device, respectively. These numbers
are assigned by the USB forum and are defined in the USB specification.
These values specify the behavior for the whole device, including all interfaces
on this device.
__u8 bInterfaceClass
__u8 bInterfaceSubClass
__u8 bInterfaceProtocol
Much like the device-specific values above, these define the class, subclass, and
protocol of the individual interface, respectively. These numbers are assigned by
the USB forum and are defined in the USB specification.
kernel_ulong_t driver_info
This value is not used to match against, but it holds information that the driver
can use to differentiate the different devices from each other in the probe callback
function to the USB driver.
As with PCI devices, there are a number of macros that are used to initialize this
structure:
USB_DEVICE(vendor, product)
Creates a struct usb_device_id that can be used to match only the specified vendor
and product ID values. This is very commonly used for USB devices that
need a specific driver.
USB_DEVICE_VER(vendor, product, lo, hi)
Creates a struct usb_device_id that can be used to match only the specified vendor
and product ID values within a version range.
USB_DEVICE_INFO(class, subclass, protocol)
Creates a struct usb_device_id that can be used to match a specific class of USB
devices.
USB_INTERFACE_INFO(class, subclass, protocol)
Creates a struct usb_device_id that can be used to match a specific class of USB
interfaces.
So, for a simple USB device driver that controls only a single USB device from a single
vendor, the struct usb_device_id table would be defined as:
/* table of devices that work with this driver */
static struct usb_device_id skel_table [ ] = {
{ USB_DEVICE(USB_SKEL_VENDOR_ID, USB_SKEL_PRODUCT_ID) },
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE (usb, skel_table);
PAGE 348
As with a PCI driver, the MODULE_DEVICE_TABLE macro is necessary to allow user-space
tools to figure out what devices this driver can control. But for USB drivers, the string
usb must be the first value in the macro.
Registering a USB Driver
The main structure that all USB drivers must create is a struct usb_driver. This
structure must be filled out by the USB driver and consists of a number of function
callbacks and variables that describe the USB driver to the USB core code:
struct module *owner
Pointer to the module owner of this driver. The USB core uses it to properly reference
count this USB driver so that it is not unloaded at inopportune moments.
The variable should be set to the THIS_MODULE macro.
const char *name
Pointer to the name of the driver. It must be unique among all USB drivers in the
kernel and is normally set to the same name as the module name of the driver. It
shows up in sysfs under /sys/bus/usb/drivers/ when the driver is in the kernel.
const struct usb_device_id *id_table
Pointer to the struct usb_device_id table that contains a list of all of the different
kinds of USB devices this driver can accept. If this variable is not set, the
probe function callback in the USB driver is never called. If you want your driver
always to be called for every USB device in the system, create a entry that sets
only the driver_info field:
static struct usb_device_id usb_ids[ ] = {
{.driver_info = 42},
{ }
};
int (*probe) (struct usb_interface *intf, const struct usb_device_id *id)
Pointer to the probe function in the USB driver. This function (described in the
section "probe and disconnect in Detail") is called by the USB core when it
thinks it has a struct usb_interface that this driver can handle. A pointer to the
struct usb_device_id that the USB core used to make this decision is also passed
to this function. If the USB driver claims the struct usb_interface that is passed
to it, it should initialize the device properly and return 0. If the driver does not
want to claim the device, or an error occurs, it should return a negative error value.
void (*disconnect) (struct usb_interface *intf)
Pointer to the disconnect function in the USB driver. This function (described in
the section "probe and disconnect in Detail") is called by the USB core when the
struct usb_interface has been removed from the system or when the driver is
being unloaded from the USB core.
PAGE 349
So, to create a value struct usb_driver structure, only five fields need to be initialized:
static struct usb_driver skel_driver = {
.owner = THIS_MODULE,
.name = "skeleton",
.id_table = skel_table,
.probe = skel_probe,
.disconnect = skel_disconnect,
};
The struct usb_driver does contain a few more callbacks, which are generally not
used very often, and are not required in order for a USB driver to work properly:
int (*ioctl) (struct usb_interface *intf, unsigned int code, void *buf)
Pointer to an ioctl function in the USB driver. If it is present, it is called when a
user-space program makes a ioctl call on the usbfs filesystem device entry associated
with a USB device attached to this USB driver. In practice, only the USB hub
driver uses this ioctl, as there is no other real need for any other USB driver to use it.
int (*suspend) (struct usb_interface *intf, u32 state)
Pointer to a suspend function in the USB driver. It is called when the device is to
be suspended by the USB core.
int (*resume) (struct usb_interface *intf)
Pointer to a resume function in the USB driver. It is called when the device is
being resumed by the USB core.
To register the struct usb_driver with the USB core, a call to usb_register_driver is
made with a pointer to the struct usb_driver. This is traditionally done in the module
initialization code for the USB driver:
static int __init usb_skel_init(void)
{
int result;
/* register this driver with the USB subsystem */
result = usb_register(&skel_driver);
if (result)
err("usb_register failed. Error number %d", result);
return result;
}
When the USB driver is to be unloaded, the struct usb_driver needs to be unregistered
from the kernel. This is done with a call to usb_deregister_driver. When this
call happens, any USB interfaces that were currently bound to this driver are disconnected,
and the disconnect function is called for them.
static void __exit usb_skel_exit(void)
{
/* deregister this driver with the USB subsystem */
usb_deregister(&skel_driver);
}
PAGE 350
probe and disconnect in Detail
In the struct usb_driver structure described in the previous section, the driver specified
two functions that the USB core calls at appropriate times. The probe function is
called when a device is installed that the USB core thinks this driver should handle;
the probe function should perform checks on the information passed to it about the
device and decide whether the driver is really appropriate for that device. The disconnect
function is called when the driver should no longer control the device for some
reason and can do clean-up.
Both the probe and disconnect function callbacks are called in the context of the USB
hub kernel thread, so it is legal to sleep within them. However, it is recommended
that the majority of work be done when the device is opened by a user if possible, in
order to keep the USB probingtime to a minimum. This is because the USB core
handles the addition and removal of USB devices within a single thread, so any slow
device driver can cause the USB device detection time to slow down and become
noticeable by the user.
In the probe function callback, the USB driver should initialize any local structures
that it might use to manage the USB device. It should also save any information that
it needs about the device to the local structure, as it is usually easier to do so at this
time. As an example, USB drivers usually want to detect what the endpoint address
and buffer sizes are for the device, as they are needed in order to communicate with
the device. Here is some example code that detects both IN and OUT endpoints of
BULK type and saves some information about them in a local device structure:
/* set up the endpoint information */
/* use only the first bulk-in and bulk-out endpoints */
iface_desc = interface->cur_altsetting;
for (i = 0; i < iface_desc->desc.bNumEndpoints; ++i) {
endpoint = &iface_desc->endpoint[i].desc;
if (!dev->bulk_in_endpointAddr &&
(endpoint->bEndpointAddress & USB_DIR_IN) &&
((endpoint->bmAttributes & USB_ENDPOINT_XFERTYPE_MASK)
= = USB_ENDPOINT_XFER_BULK)) {
/* we found a bulk in endpoint */
buffer_size = endpoint->wMaxPacketSize;
dev->bulk_in_size = buffer_size;
dev->bulk_in_endpointAddr = endpoint->bEndpointAddress;
dev->bulk_in_buffer = kmalloc(buffer_size, GFP_KERNEL);
if (!dev->bulk_in_buffer) {
err("Could not allocate bulk_in_buffer");
goto error;
}
}
if (!dev->bulk_out_endpointAddr &&
!(endpoint->bEndpointAddress & USB_DIR_IN) &&
((endpoint->bmAttributes & USB_ENDPOINT_XFERTYPE_MASK)
PAGE 351
= = USB_ENDPOINT_XFER_BULK)) {
/* we found a bulk out endpoint */
dev->bulk_out_endpointAddr = endpoint->bEndpointAddress;
}
}
if (!(dev->bulk_in_endpointAddr && dev->bulk_out_endpointAddr)) {
err("Could not find both bulk-in and bulk-out endpoints");
goto error;
}
This block of code first loops over every endpoint that is present in this interface and
assigns a local pointer to the endpoint structure to make it easier to access later:
for (i = 0; i < iface_desc->desc.bNumEndpoints; ++i) {
endpoint = &iface_desc->endpoint[i].desc;
Then, after we have an endpoint, and we have not found a bulk IN type endpoint
already, we look to see if this endpoint's direction is IN. That can be tested by seeing
whether the bitmask USB_DIR_IN is contained in the bEndpointAddress endpoint variable.
If this is true, we determine whether the endpoint type is bulk or not, by first
maskingoff the bmAttributes variable with the USB_ENDPOINT_XFERTYPE_MASK bitmask,
and then checking if it matches the value USB_ENDPOINT_XFER_BULK:
if (!dev->bulk_in_endpointAddr &&
(endpoint->bEndpointAddress & USB_DIR_IN) &&
((endpoint->bmAttributes & USB_ENDPOINT_XFERTYPE_MASK)
= = USB_ENDPOINT_XFER_BULK)) {
If all of these tests are true, the driver knows it found the proper type of endpoint
and can save the information about the endpoint that it will later need to communicate
over it in a local structure:
/* we found a bulk in endpoint */
buffer_size = endpoint->wMaxPacketSize;
dev->bulk_in_size = buffer_size;
dev->bulk_in_endpointAddr = endpoint->bEndpointAddress;
dev->bulk_in_buffer = kmalloc(buffer_size, GFP_KERNEL);
if (!dev->bulk_in_buffer) {
err("Could not allocate bulk_in_buffer");
goto error;
}
Because the USB driver needs to retrieve the local data structure that is associated
with this struct usb_interface later in the lifecycle of the device, the function
usb_set_intfdata can be called:
/* save our data pointer in this interface device */
usb_set_intfdata(interface, dev);
This function accepts a pointer to any data type and saves it in the struct usb_interface
structure for later access. To retrieve the data, the function usb_get_intfdata should
be called:
struct usb_skel *dev;
struct usb_interface *interface;
PAGE 352
int subminor;
int retval = 0;
subminor = iminor(inode);
interface = usb_find_interface(&skel_driver, subminor);
if (!interface) {
err ("%s - error, can't find device for minor %d",
__FUNCTION__, subminor);
retval = -ENODEV;
goto exit;
}
dev = usb_get_intfdata(interface);
if (!dev) {
retval = -ENODEV;
goto exit;
}
usb_get_intfdata is usually called in the open function of the USB driver and again in
the disconnect function. Thanks to these two functions, USB drivers do not need to
keep a static array of pointers that store the individual device structures for all current
devices in the system. The indirect reference to device information allows an
unlimited number of devices to be supported by any USB driver.
If the USB driver is not associated with another type of subsystem that handles the
user interaction with the device (such as input, tty, video, etc.), the driver can use the
USB major number in order to use the traditional char driver interface with user
space. To do this, the USB driver must call the usb_register_dev function in the probe
function when it wants to register a device with the USB core. Make sure that the
device and driver are in a proper state to handle a user wanting to access the device
as soon as this function is called.
/* we can register the device now, as it is ready */
retval = usb_register_dev(interface, &skel_class);
if (retval) {
/* something prevented us from registering this driver */
err("Not able to get a minor for this device.");
usb_set_intfdata(interface, NULL);
goto error;
}
The usb_register_dev function requires a pointer to a struct usb_interface and a
pointer to a struct usb_class_driver. This struct usb_class_driver is used to define
a number of different parameters that the USB driver wants the USB core to know
when registering for a minor number. This structure consists of the following variables:
char *name
The name that sysfs uses to describe the device. A leading pathname, if present,
is used only in devfs and is not covered in this book. If the number of the device
needs to be in the name, the characters %d should be in the name string. For
PAGE 353
example, to create the devfs name usb/foo1 and the sysfs class name foo1, the
name string should be set to usb/foo%d.
struct file_operations *fops;
Pointer to the struct file_operations that this driver has defined to use to register
as the character device. See Chapter 3 for more information about this structure.
mode_t mode;
The mode for the devfs file to be created for this driver; unused otherwise. A typical
setting for this variable would be the value S_IRUSR combined with the value
S_IWUSR, which would provide only read and write access by the owner of the
device file.
int minor_base;
This is the start of the assigned minor range for this driver. All devices associated
with this driver are created with unique, increasing minor numbers beginning with
this value. Only 16 devices are allowed to be associated with this
driver at any one time unless the CONFIG_USB_DYNAMIC_MINORS configuration
option has been enabled for the kernel. If so, this variable is ignored, and all
minor numbers for the device are allocated on a first-come, first-served manner.
It is recommended that systems that have enabled this option use a program
such as udev to manage the device nodes in the system, as a static /dev tree will
not work properly.
When the USB device is disconnected, all resources associated with the device
should be cleaned up, if possible. At this time, if usb_register_dev has been called to
allocate a minor number for this USB device during the probe function, the function
usb_deregister_dev must be called to give the minor number back to the USB core.
In the disconnect function, it is also important to retrieve from the interface any data
that was previously set with a call to usb_set_intfdata. Then set the data pointer in
the struct usb_interface structure to NULL to prevent any further mistakes in accessing
the data improperly:
static void skel_disconnect(struct usb_interface *interface)
{
struct usb_skel *dev;
int minor = interface->minor;
/* prevent skel_open( ) from racing skel_disconnect( ) */
lock_kernel( );
dev = usb_get_intfdata(interface);
usb_set_intfdata(interface, NULL);
/* give back our minor */
usb_deregister_dev(interface, &skel_class);
unlock_kernel( );
PAGE 354
/* decrement our usage count */
kref_put(&dev->kref, skel_delete);
info("USB Skeleton #%d now disconnected", minor);
}
Note the call to lock_kernel in the previous code snippet. This takes the big kernel
lock, so that the disconnect callback does not encounter a race condition with the
open call when trying to get a pointer to the correct interface data structure. Because
the open is called with the bigkernel lock taken, if the disconnect also takes that same
lock, only one portion of the driver can access and then set the interface data pointer.
Just before the disconnect function is called for a USB device, all urbs that are currently
in transmission for the device are canceled by the USB core, so the driver does
not have to explicitly call usb_kill_urb for these urbs. If a driver tries to submit a urb
to a USB device after it has been disconnected with a call to usb_submit_urb, the submission
will fail with an error value of -EPIPE.
Submitting and Controlling a Urb
When the driver has data to send to the USB device (as typically happens in a driver's
write function), a urb must be allocated for transmitting the data to the device:
urb = usb_alloc_urb(0, GFP_KERNEL);
if (!urb) {
retval = -ENOMEM;
goto error;
}
After the urb is allocated successfully, a DMA buffer should also be created to send
the data to the device in the most efficient manner, and the data that is passed to the
driver should be copied into that buffer:
buf = usb_buffer_alloc(dev->udev, count, GFP_KERNEL, &urb->transfer_dma);
if (!buf) {
retval = -ENOMEM;
goto error;
}
if (copy_from_user(buf, user_buffer, count)) {
retval = -EFAULT;
goto error;
}
Once the data is properly copied from the user space into the local buffer, the urb
must be initialized correctly before it can be submitted to the USB core:
/* initialize the urb properly */
usb_fill_bulk_urb(urb, dev->udev,
usb_sndbulkpipe(dev->udev, dev->bulk_out_endpointAddr),
buf, count, skel_write_bulk_callback, dev);
urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
PAGE 355
Now that the urb is properly allocated, the data is properly copied, and the urb is
properly initialized, it can be submitted to the USB core to be transmitted to the
device:
/* send the data out the bulk port */
retval = usb_submit_urb(urb, GFP_KERNEL);
if (retval) {
err("%s - failed submitting write urb, error %d", __FUNCTION__, retval);
goto error;
}
After the urb is successfully transmitted to the USB device (or something happens in
transmission), the urb callback is called by the USB core. In our example, we initialized
the urb to point to the function skel_write_bulk_callback, and that is the function
that is called:
static void skel_write_bulk_callback(struct urb *urb, struct pt_regs *regs)
{
/* sync/async unlink faults aren't errors */
if (urb->status &&
!(urb->status = = -ENOENT ||
urb->status = = -ECONNRESET ||
urb->status = = -ESHUTDOWN)) {
dbg("%s - nonzero write bulk status received: %d",
__FUNCTION__, urb->status);
}
/* free up our allocated buffer */
usb_buffer_free(urb->dev, urb->transfer_buffer_length,
urb->transfer_buffer, urb->transfer_dma);
}
The first thing the callback function does is check the status of the urb to determine
if this urb completed successfully or not. The error values, -ENOENT, -ECONNRESET, and
-ESHUTDOWN are not real transmission errors, just reports about conditions accompanying a
successful transmission. (See the list of possible errors for urbs detailed in the
section "struct urb.") Then the callback frees up the allocated buffer that was
assigned to this urb to transmit.
It's common for another urb to be submitted to the device while the urb callback
function is running. This is useful when streaming data to a device. Remember that
the urb callback is running in interrupt context, so it should do any memory allocation,
hold any semaphores, or do anything else that could cause the process to sleep.
When submitting a urb from within a callback, use the GFP_ATOMIC flag to tell the
USB core to not sleep if it needs to allocate new memory chunks during the submission
process.
PAGE 356
USB Transfers Without Urbs
Sometimes a USB driver does not want to go through all of the hassle of creating a
struct urb, initializing it, and then waiting for the urb completion function to run,
just to send or receive some simple USB data. Two functions are available to provide
a simpler interface.
usb_bulk_msg
usb_bulk_msg creates a USB bulk urb and sends it to the specified device, then waits
for it to complete before returning to the caller. It is defined as:
int usb_bulk_msg(struct usb_device *usb_dev, unsigned int pipe,
void *data, int len, int *actual_length,
int timeout);
The parameters of this function are:
struct usb_device *usb_dev
A pointer to the USB device to send the bulk message to.
unsigned int pipe
The specific endpoint of the USB device to which this bulk message is to be sent.
This value is created with a call to either usb_sndbulkpipe or usb_rcvbulkpipe.
void *data
A pointer to the data to send to the device if this is an OUT endpoint. If this is
an IN endpoint, this is a pointer to where the data should be placed after being
read from the device.
int len
The length of the buffer that is pointed to by the data parameter.
int *actual_length
A pointer to where the function places the actual number of bytes that have
either been transferred to the device or received from the device, depending on
the direction of the endpoint.
int timeout
The amount of time, in jiffies, that should be waited before timing out. If this
value is 0, the function waits forever for the message to complete.
If the function is successful, the return value is 0; otherwise, a negative error number
is returned. This error number matches up with the error numbers previously
described for urbs in the section "struct urb." If successful, the actual_length parameter
contains the number of bytes that were transferred or received from this message.
The following is an example of using this function call:
/* do a