p1: Cush - notes

-Develop a simple job control shell

-Shell receives line-by-line input
    -if input = built-in command, shell will execute command
    -else, shell will interpret input as the name of an external program to be executed
        -forks a new child process, executes the program in the context of the child
    -shell waits for command to complete before reading the next command
        -unless, user appends '&', in which case the command is started immediately.
    -the running commands are called "background jobs"
    -the commands the shell waits for before processing new inputs are called "foreground jobs"

-Shell provides job control.
    -User can inerrupt foreground jobs, and send foreground jobs to the background.
    -Shell will run zero or more background jobs, and zero or one foreground jobs.
        -To elaborate, foreground jobs will go through one at a time, shell will wait for it to
        complete before printing another prompt and reading next command.
    -Shell provides status changes of the jobs it manages. Jobs may exit, terminate to signal, etc.

MINIMUM REQUIREMENTS:
    -At a minimum, we expect that your shell has the ability to start foreground and background jobs
    and implements the built-in commands ‘jobs,’ ‘fg,’ ‘bg,’ ‘kill,’ ’exit,’ and ‘stop.
    -To fully complete this project, students need to be able to implement pipes, I/O, Redirection and managing controlling
    the terminal to ensure jobs get access to terminal wheen needed.

SIGCHLD:
-Shell can be running multiple jobs. Shell does NOT and cannot know what these programs do.
    -SIGCHLD is a signal sent from OS to inform the shell of the changing status of a job.
    -Cush will need to catch SIGCHLD. Difficult though, because Cush has no knowledge of what these processes are doing,
    impossible to predict when a child will exit, therefore impossible to predict when SIGCHLD comes.
    -Ex: New job is added to data structure after a child process is forked. In order to stop the situation in which SIGCHLD
    arrives while the shell is adding the job, the parent should block SIGCHLD until it completed adding job to list data structure.
    -use helper functions in signal_support.c to block/unblock signals.

Process groups:
-Job may involve multiple processes.
    -Ex: ls | grep filename requires the shell to start 2 processes: ls and grep
    -Each process in UNIX is part of a group. Mainly for the purpose of signal delivery (kill, waitpid)
    and waiting for processes to finish (think child processes).
-The shell must create a new process group for each job and make sure that all processes
that will be created for this job become members of this group.
-Each process group has a leader, one of the processes in the group. Process calls setpgid(0, 0) to create a 
new group with itself as the leader.
-Child processes inherit the process group of their parent process, but they can form their own group
or their parent process can place them in a new group via setpgid()

Terminal Access:
-Running multiple processes on same terminal creates sharing issue,
imagine two processes reading from terminal at same time, which gets the input?
-Foreground process group!
-If a process in a non-foreground process group attempts to perform an operation that requires exclusive access to terminal,
it is sent a signal: SIGTTOU or SIGTTIN. Default action taken is to suspend the processes in that group. Call waitpid(), WIFSTOPPED(status)
will be true.
-To allow these processes to continue, their process group must be made the foreground process group of the controlling terminal
via a call to tcsetpgrp(), and sent a SIGCONT signal. This occurs in Cush when 'fg' built-in command is issued.
-User input signals, SIGINT, SIGSTP, are sent to a terminal's foreground process group. Occurs automatically by OS, not an action
the shell takes.
-Delivering such signals to an entire process group makes it so all processes running in a job will be terminated or other.
-Shell must arrange for these process groups to exist and be populated with the correct processes. As well as, inform the OS which
process group the user intends to run in the foreground.

Terminal State:
-Cush puts terminal in 'cooked' mode, using readline library which requires user to end line with enter.

  - foreground.py
  - cmdfail_and_exit_test.py
  - ctrl-z_test.py
  - ctrl-c_test.py
  - fg_test.py
  - jobs_test.py
  - stop_test.py
  - bg_test.py
  - reap_on_child_termination_test.py
  - kill_test.py
  - signal_test.py (SIGINT, SIGTSTP, SIGCHLD)

   - io_in_test.py
  - io_out_test.py
  - single_pipe_test.py
  - single_pipe_test_stderr.py
  - io_append_test.py
  - multi_pipe_test.py
  - io_and_pipes.py
  - pipe_job_cntl.py
  - exclusive_access_test.py
  - multiple_pipes.py
  - test_termstate.py
  - test_termstate2.py

  -extra_built-in #1
  -extra_built-in #2
---------------------------------------------------------------------------------------------
Cush_help_session_fall_2022:

Shell Concepts: 
-Shell is a command intepreter, executable with no GUI, terminal is the GUI
-Terminal (the front end of our shell)

-Shell waits for user input
-Shell interprets the command
-Forks a process
-If its a foreground parent, waits for child to finish. Else, parent
repeats the process again. (Child executes command)

Foreground:
-1 Process group at a time, has access to terminal
-ex sleep 100

Background:
-Does not have terminal access
-Using '&' sends command to background to run

Process groups:
-Each job is its own process group
    -Each command within job have same PGID
    -posix_spawn is how to create a new process
-Jobs are deleted when they are completed
    -Try not to prematurely delete jobs, use wait_for_job() and comment above it

sleep 20 | sleep 20 | sleep 20 & 

Each sleep has a different process ID, but all have the same PGID and PPID

POSIX spawn:
-replaces fork() + exec() entirely
-code is linear, rathing than handling multiple processes in if/else statements like ex1

Example code
{
posix_spawn_file_actions_t child_file_attr;
posix_spawnattr_t child_spawn_attr;

posix_spawnattr_int(&child_file_attr);
posix_spawn_file_Actions_init(&child_file_attr);

//setup for attributes

posix_spawnp(/*pid*/, /*program*/, &child_file_attr, &child_spawn_attr, /*program arguments*/, environ)
}

Process groups - posix_spawnattr_getpgroup()
Terminal control - posix_spawnattr_tcsetpgrp_np()
Piping - posix_spawn_file_actions_adddup2()

man page: spawn.h

Built-in commands:
-Commands that are defined within the program by you
    -No need to fork off and execute an external program

Built-ins behind the scenes:
-NO making on new process, no forking, etc. After execution, shell repeats.

I/O Piping: 
The Shell will fork off a child process to execute each command in a pipeline.
Processes will wait on previous process, final process outputs to terminal.
STDIN and STDOUT for processes are joined to create the pipeline.

I/O Redirection: >, >>, <

Signal handling:
-Signals sent by the OS. We need to catch them and deal with them.
-Shells can handle signals sent to them
    -SIGINT (Ctrl + C)
    -SIGTSTP(Ctrl + Z)
    -SIGCHLD (when a child process terminates)
SIGCHLD notifies parent when child process finishes. Use WIF* macros to decode
the status sent by wait* calls to discover what state a process changed to, and how
it happened.

; --> parser. It splits a command line input to two different pipelines

Data structures:

1). ast_command_line: sleep 10 &; echo hi | rev
1).-> 2) ast_pipeline: sleep 10 &
1).-> 3) ast_pipeline: echo hi | rev

2). -> 4). ast_command: sleep 10
3). -> 5). ast_command: echo hi
3). -> 6). ast_command: rev

Hard to understand but,
ast_command_line is top of the data structure abstraction
ast_pipeline is directly under ast_command_line, (;)
ast_command is directly under ast_pipeline (|, >, <, etc)

Linked_List or list.c:

Sentinel node points to list_elem. List_elem point to other list_elem.

 struct list_elem{
    struct list_elem * prev;
    struct list_elem * next;
 }

//retrieve data from a struct list_elem by using the list_entry macro
 struct ast_command * cmd = list_entry(e, struct ast_command, elem);

An example in Cush, the job struct.
Adding list_elem to a struct allows this struct to be added to a list

Warnings ab list: 
-DON'T use same list_elem for multiple lists
-Edit an element while iterating, naive loop to remove elements in a list will fail!
-Forget to list_init()

Bad idea for Cush: this will corrupt the list and throw errors

for(list_elem in list)
{

    if(condition)
    {
        list_remove(currElem);
    }
}

List for loop (this needs to be in the project):

for(struct list_elem * e = list_begin(&pipe->commands); e != list_end(&pipe->commands);)
{
     struct ast_command * cmd = list_entry(e, struct ast_command, elem);
     e = list_remove(e); //Acts as the iterator; stores next element into each
     ast_command_free(cmd);
}

free(pipe);

Test Driver:
cd src/..tests/stdriver.py [options] 
(options include -b for basic tests, -a advance tests, -h options, etc)
Write additional .tst files that tests custom builtins

Design Document: README.txt (just wrote the skeleton of this document)

GDB tips!!:

(gdb) set follow-fork-mode <mode> 
mode = 'parent' or 'child'
parent ignores child process and continue debugging the parent
child begins debugging the child process when fork() is called

Retain debugger control after fork:
(gdb) set detach-on-fork <mode>

Nice place to start?
Implement basic functionality
Implement 'jobs'
Implement 'exit' (1 line of code :D! )
If you do not implement the basic functionality, the advanced tests
will always give you a 0, failing the tests.

Vast majority of code goes into:
main()
handle_child_status()
helper methods, helper functions
adding/editing structs, like adding new fields to job, etc

do not go crazy removing code/messing with other files. Most given code is useful and
doesn't need changes. TA code was roughly ~700-800 lines long.