checklist-tester/pub/dissertation/chapters/results.tex

\documentclass[../dissertation.tex]{subfiles}

\begin{document}
\section{Final Prototype}
\subsection{Formal Model}
\begin{itemize}
  \item The model is mostly designed to imitate a Boeing 737-800,
    as the types modelled, have user inputs which are different from
    other aircraft types
    \begin{itemize}
      \item For example, the Airbus A320 has push buttons whereas
      they are not there on the 737-800
      \item However, further user input types could be added to the model
        and as a result, further aircraft types could have their
        procedures run through the formal model
    \end{itemize}
  \item The \lstinline|Procedure| type makes sure that the items on
    the procedure is completed in order, and if a step is missed,
    that would result in an invariant failure, resulting in the
    checklist test failing
  % TODO write more
  % Stuff like:
  % - How items are handled
  % - Functions such as auto complete or step by step
  % - Why Aircraft?
\end{itemize}

\subsection{Checklist Tester}
\begin{itemize}
  \item The main features of GUI have been completed, it has all the sections desired
  \begin{itemize}
    \item Projects can be created to split up different aircraft
      or revisions of checklists
    \item Procedures can be created and tested
    \item These procedures get tested in the flight simulator automatically
      and gives the results of how the procedure has been doing in
      real time
  \end{itemize}
\end{itemize}

\subsection{Setting up Tests}
\begin{itemize}
  \item Each test is set up by defining each action in the procedure,
    on the Procedure screen
  \item To be able to define each action is supposed to do, it uses
    the Dataref variables in X-Plane, which is what stores the state
    of the aircraft. Each switch has their own unique Dataref
  \item In the checklist tester then, each action asks for a
    Dataref and a desired goal value
  \item Some Datarefs are read only, but there are other Datarefs
    for the item desired, but are only \enquote{command}s, which
    can only be called and not have its value changed; this can be
    run by setting the desired goal value to be -988 (because XPC uses that value)
\end{itemize}

\subsubsection{Running Tests}
\begin{itemize}
  \item Tests are run by connecting to the flight simulator, X-Plane
  \item The tester goes through each action in the procedure one by one
    and waits for the current action to complete before proceeding on to
    the next one
  \item The checklist tester is not advanced enough to be able to control
    fly the aircraft; hence the tester would be able to engage autopilot
    first, or control the aircraft themselves, where the checklist tester
    would be acting like a first officer
\end{itemize}

\subsubsection{Storing Test Results}
\begin{itemize}
  \item There is a database storing the results of each of the tests
  \item Each tests store
  \begin{description}
    \item[Time taken] for each of the actions in the procedure to complete
    \item[Start state] for the state that the action in the procedure was at
    \item[End state] for the state that the action in the procedure finished
      the item at
    \item[Overall test time] Stores the time taken from when the test started
      to when the test ended
  \end{description}
  \item This gives feedback/statistics for the checklist designers
    to find areas of improvement on the procedure, such as one action
    in the procedure taking too long, may point out a potential flaw
    to the designer and as a result aid finding potential alternative
    options for that step in the procedure
\end{itemize}

\section{Problems Found}


\section{LOC?}
% TODO I don't know what LOC meant, was it Language of Choice?


\section{Reflection}
\subsection{Planning}
\subsubsection{Gantt Chart}
Used Gantt chart to create a plan for what would be needed from this project

\textbf{Pros:}
\begin{itemize}
  \item Was useful for the first part because it set expectations
    of what was needed and how much time there was to complete them
  \item Helped visualize the different components of the project
  \item Helped in the beginning being accompanied by a Kanban in Leantime\footnote{\url{https://leantime.io/}}
\end{itemize}

\textbf{Cons:}
\begin{itemize}
  \item Was not detailed enough, and a design document would have been useful
    to accompany the Gantt chart for each section
  \item The lack of detail was not helpful when falling behind as having
    attention deficit hyperactivity disorder (ADHD) 
    added to the burden of feeling like each section was a massive project
  \item Leantime's claim for being \enquote{built with ADHD [\ldots] in mind}
    felt misleading as navigating through it felt worse than using the front page
    of Stack Overflow\footnote{\url{https://stackoverflow.com/}}
  \item Todoist\footnote{\url{https://todoist.com/}} was a good alternative though
\end{itemize}

\subsubsection{GUI Design}
Figma was very useful in implementations as

\textbf{Pros:}
\begin{itemize}
  \item It helped with timing and knowing what to do
  \item Made things feel manageable as it was split up to different sections
  \item Meant features will not be forgotten
\end{itemize}

\textbf{Cons:}
\begin{itemize}
  \item Certain features being too simple and annoying to use
  \item A bit of a learning curve for using other components, compared
    to using plugins
\end{itemize}

\subsection{Implementation}

\subsubsection{Checklist Tester}
\begin{itemize}
  \item Implementing the GUI was useful to split up the sections required
    for the project and having a goal for what to be done
  \item However, a bit too much time was spent on creating a GUI when it
    could have been used for development
  \item It was useful for motivational reasons to feel like something
    materialistic has been produced rather than something theoretical
  \item Was originally intended to be used to interact with custom
    plugin for X-Plane as it would have been difficult otherwise
\end{itemize}

\subsubsection{Connecting to the Flight Simulator}
\begin{itemize}
  \item Would have been more useful to search a bit further if there was
    another plugin available, as found Dataref Editor on the X-Plane docs,
    so could have looked for a similar plugin for connecting to X-Plane
  \item At first spent about a week developing a C++ X-Plane plugin from scratch,
    requiring to figure out sockets
  \item At the same time finding out XPC exists and having wasted that time
  \item However, it did teach me more about understanding how sockets work and
    more about C++ and setting up a project with CMake and adding packages with
    vcpkg
\end{itemize}

\section{Time Spent}
\begin{itemize}
  \item Time spent was recorded using Wakatime, other than time spent
    researching, which had to be recorded manually, using Leantime
  \item The time spent on GUI is also time spent on connecting other tools
    such as the VDMJ wrapper, XPC, and the database
\end{itemize}
\begin{figure}[!htp]
  \centering
  % TODO make it show how many hours were spent instead of percentages
  \begin{tikzpicture}
    \pie[sum=auto, text=legend]{%
      40/GUI,
      11.5/Database,
      2/Unit Tests,
      14.5/Configuring Connector,
      15.5/Formal Modelling,
      13/VDMJ Wrapper,
      6/Packaging XPC,
      32/Research%
    }
    % Used to make the 2 go to the outside of the pie chart 
    \pie[sum=134.5, hide number]{40/, 11.5/, 2/2}
    \pie[sum=134.5]{40/, 11.5/}
  \end{tikzpicture}
  \caption{Time spent on sections of project (in hours)}
\end{figure}

\end{document}