cpuworker.c File Reference

Implements a farm of 'CPU worker' processes to perform CPU-intensive tasks in another thread or process, to not interrupt the main thread. More...

#include "or.h"

Defines

#define MAX_CPUWORKERS   16
#define MIN_CPUWORKERS   1
#define TAG_LEN   10
#define LEN_ONION_RESPONSE   (1+TAG_LEN+ONIONSKIN_REPLY_LEN+CPATH_KEY_MATERIAL_LEN)
#define CPUWORKER_BUSY_TIMEOUT   (60*60*12)

Functions

static void cpuworker_main (void *data) ATTR_NORETURN
static int spawn_cpuworker (void)
static void spawn_enough_cpuworkers (void)
static void process_pending_task (connection_t *cpuworker)
void cpu_init (void)
int connection_cpu_finished_flushing (connection_t *conn)
static void tag_pack (char *tag, uint64_t conn_id, circid_t circ_id)
static void tag_unpack (const char *tag, uint64_t *conn_id, circid_t *circ_id)
void cpuworkers_rotate (void)
int connection_cpu_reached_eof (connection_t *conn)
int connection_cpu_process_inbuf (connection_t *conn)
static void cull_wedged_cpuworkers (void)
int assign_onionskin_to_cpuworker (connection_t *cpuworker, or_circuit_t *circ, char *onionskin)

Variables

static int num_cpuworkers = 0
static int num_cpuworkers_busy = 0
static time_t last_rotation_time = 0


Detailed Description

Implements a farm of 'CPU worker' processes to perform CPU-intensive tasks in another thread or process, to not interrupt the main thread.

Right now, we only use this for processing onionskins.


Define Documentation

#define CPUWORKER_BUSY_TIMEOUT   (60*60*12)

How long should we let a cpuworker stay busy before we give up on it and decide that we have a bug or infinite loop? This value is high because some servers with low memory/cpu sometimes spend an hour or more swapping, and Tor starves.

Referenced by cull_wedged_cpuworkers().

#define LEN_ONION_RESPONSE   (1+TAG_LEN+ONIONSKIN_REPLY_LEN+CPATH_KEY_MATERIAL_LEN)

How many bytes are sent from the cpuworker back to tor?

Referenced by connection_cpu_process_inbuf(), and cpuworker_main().

#define MAX_CPUWORKERS   16

The maximum number of cpuworker processes we will keep around.

Referenced by spawn_enough_cpuworkers().

#define MIN_CPUWORKERS   1

The minimum number of cpuworker processes we will keep around.

Referenced by spawn_enough_cpuworkers().

#define TAG_LEN   10

The tag specifies which circuit this onionskin was from.

Referenced by assign_onionskin_to_cpuworker(), connection_cpu_process_inbuf(), and cpuworker_main().


Function Documentation

int assign_onionskin_to_cpuworker ( connection_t cpuworker,
or_circuit_t circ,
char *  onionskin 
)

Try to tell a cpuworker to perform the public key operations necessary to respond to onionskin for the circuit circ.

If cpuworker is defined, assert that he's idle, and use him. Else, look for an idle cpuworker and use him. If none idle, queue task onto the pending onion list and return. Return 0 if we successfully assign the task, or -1 on failure.

References or_connection_t::_base, CONN_TYPE_CPUWORKER, connection_get_by_type_state(), CPUWORKER_STATE_BUSY_ONION, CPUWORKER_STATE_IDLE, cull_wedged_cpuworkers(), connection_t::global_identifier, LD_OR, num_cpuworkers, num_cpuworkers_busy, onion_pending_add(), or_circuit_t::p_circ_id, or_circuit_t::p_conn, spawn_enough_cpuworkers(), connection_t::state, TAG_LEN, tag_pack(), connection_t::timestamp_lastwritten, tor_assert, and tor_free.

Referenced by command_process_create_cell(), and process_pending_task().

int connection_cpu_finished_flushing ( connection_t conn  ) 

Called when we're done sending a request to a cpuworker.

References CONN_TYPE_CPUWORKER, connection_stop_writing(), tor_assert, and connection_t::type.

Referenced by connection_finished_flushing().

int connection_cpu_process_inbuf ( connection_t conn  ) 

int connection_cpu_reached_eof ( connection_t conn  ) 

If the cpuworker closes the connection, mark it as closed and spawn a new one as needed.

References CPUWORKER_STATE_IDLE, LD_GENERAL, num_cpuworkers, num_cpuworkers_busy, spawn_enough_cpuworkers(), and connection_t::state.

Referenced by connection_reached_eof().

void cpu_init ( void   ) 

Initialize the cpuworker subsystem.

References cpuworkers_rotate().

Referenced by do_main_loop().

static void cpuworker_main ( void *  data  )  [static]

Implement a cpuworker. 'data' is an fdarray as returned by socketpair. Read and writes from fdarray[1]. Reads requests, writes answers.

Request format: Task type [1 byte, always CPUWORKER_TASK_ONION] Opaque tag TAG_LEN Onionskin challenge ONIONSKIN_CHALLENGE_LEN Response format: Success/failure [1 byte, boolean.] Opaque tag TAG_LEN Onionskin challenge ONIONSKIN_REPLY_LEN Negotiated keys KEY_LEN*2+DIGEST_LEN*2

(Note: this _should_ be by addr/port, since we're concerned with specific connections, not with routers (where we'd use identity).)

References crypto_free_pk_env(), crypto_thread_cleanup(), dup_onion_keys(), handle_signals(), LD_BUG, LD_OR, LEN_ONION_RESPONSE, onion_skin_server_handshake(), read_all(), spawn_exit(), TAG_LEN, tor_assert, tor_close_socket(), tor_free, tor_free_all(), and write_all().

Referenced by spawn_cpuworker().

void cpuworkers_rotate ( void   ) 

Called when the onion key has changed and we need to spawn new cpuworkers. Close all currently idle cpuworkers, and mark the last rotation time as now.

References CONN_TYPE_CPUWORKER, connection_get_by_type_state(), CPUWORKER_STATE_IDLE, get_options(), last_rotation_time, num_cpuworkers, server_mode(), and spawn_enough_cpuworkers().

Referenced by cpu_init(), do_hup(), options_act(), and run_scheduled_events().

static void cull_wedged_cpuworkers ( void   )  [static]

We have a bug that I can't find. Sometimes, very rarely, cpuworkers get stuck in the 'busy' state, even though the cpuworker process thinks of itself as idle. I don't know why. But here's a workaround to kill any cpuworker that's been busy for more than CPUWORKER_BUSY_TIMEOUT.

References CONN_TYPE_CPUWORKER, CPUWORKER_BUSY_TIMEOUT, CPUWORKER_STATE_BUSY_ONION, get_connection_array(), LD_BUG, num_cpuworkers, and num_cpuworkers_busy.

Referenced by assign_onionskin_to_cpuworker().

static void process_pending_task ( connection_t cpuworker  )  [static]

Take a pending task from the queue and assign it to 'cpuworker'.

References assign_onionskin_to_cpuworker(), LD_OR, onion_next_task(), and tor_assert.

Referenced by connection_cpu_process_inbuf().

static int spawn_cpuworker ( void   )  [static]

static void spawn_enough_cpuworkers ( void   )  [static]

If we have too few or too many active cpuworkers, try to spawn new ones or kill idle ones.

References get_options(), LD_GENERAL, MAX_CPUWORKERS, MIN_CPUWORKERS, num_cpuworkers, or_options_t::NumCpus, and spawn_cpuworker().

Referenced by assign_onionskin_to_cpuworker(), connection_cpu_process_inbuf(), connection_cpu_reached_eof(), and cpuworkers_rotate().

static void tag_pack ( char *  tag,
uint64_t  conn_id,
circid_t  circ_id 
) [static]

Pack global_id and circ_id; set *tag to the result. (See note on cpuworker_main for wire format.)

References set_uint16(), and set_uint64().

Referenced by assign_onionskin_to_cpuworker().

static void tag_unpack ( const char *  tag,
uint64_t *  conn_id,
circid_t circ_id 
) [static]

Unpack tag into addr, port, and circ_id.

References get_uint16(), and get_uint64().

Referenced by connection_cpu_process_inbuf().


Variable Documentation

time_t last_rotation_time = 0 [static]

We need to spawn new cpuworkers whenever we rotate the onion keys on platforms where execution contexts==processes. This variable stores the last time we got a key rotation event.

Referenced by connection_cpu_process_inbuf(), and cpuworkers_rotate().

int num_cpuworkers = 0 [static]

int num_cpuworkers_busy = 0 [static]

How many of the running cpuworkers have an assigned task right now.

Referenced by assign_onionskin_to_cpuworker(), connection_cpu_process_inbuf(), connection_cpu_reached_eof(), and cull_wedged_cpuworkers().


Generated on Tue May 25 00:30:42 2010 for tor by  doxygen 1.5.6