Algorithms: General Comments

Some techniques I've been studying along with an assortment of answers from or links to a variety of practice/testing sites.

All ranked Solved Examples are top 50% time-complexity or top 50% space-complexity for at least one valid run (- given the significant variance across runs. Note: this applies primarily to LeetCode -) and original unless otherwise noted. (I'm flagging those for further review.)

Profiles

Blog

https://www.thoughtscript.io/

Resume

Flashcards

https://render-static-fs.onrender.com/flash_cards_2023.html

Note:

Use the `?topics=` HTTP parameter to narrow the displayed flash cards down. 

Multiple topics can be selected at a time:

* `?topics=java,javascript`
* `?topics=sql,ruby`

Algorithms: Big O Notation

Time Complexity

By convention, Average Big O Time Complexity almost always indicates the worst-case time complexity. Time Complexity is almost always measured using Big O Notation.

Also, it’s standard practice to drop multiples e.g. - O(2n) is usually just written as O(n).

Big O Notation

Given an input n, calculate, a priori, abstracting away certain physical implementation specifics, the time it takes to execute an algorithm relative to n. By convention, Big O Notation typically measures only the worst-case time complexity (upper bound).

Formal Definition

The best formal definition I’ve found:

Where f, g are functions from N to the positive reals, R+; and
Where O(g) is a set of functions from N to the positive reals, R+; and
Where ∈ stands for the elemental inclusion operator:
- f ∈ O(g) means: asymptotically and ignoring constant factors, f does not grow more quickly than g over time or input size.
- f ∈ O(g) means: asymptotically and ignoring constant factors, g is the worst-case time complexity for f over time or input size.

Refer to: http://www.uni-forst.gwdg.de/~wkurth/cb/html/cs1_v06.pdf

Constant Time

Invariant time-complexity

O(1)
Example: f = n => 1
Example: just returning a constant.

Linear Time

Time-complexity is directly proportional to (the number of inputs) n.

O(n)
Example: f = n => { v = 0; for (i= 0 ; i < n ; i++) { v++ }; return v }
Example: un-nested looping once or more through an array.

Logarithmic Time

Time-complexity decreases or increases by a respective decreasing or increasing amount of change as the algorithm continues to run.

O(log(n))
Example: f = n => { for (i = 0 ; i < n ; i++) { i = i * 2 }; return 1 }
Example: recursively halving a number.

Logarithms

log specifies how much, y, the base, b, is raised to in order to obtain some x.

logᵦ(x) = y
(log base b x = y) if and only if (b x = y)
e.g. - (log base 2 64 = 6) if and only if (2⁶ = 64)

log(x) = y is written as shorthand for an assumed or suppressed base b.

e.g. - log₁₀(x) = y
Here (Big O notation), the assumed base is binary or 2.

Quadratic Time

Time-complexity increases exponentially with respect to (the number of inputs) n.

O(nˣ) | x > 1
Example: f = n => { v = 0; for (i = 0 ; i < n ; i++) { for (j = 0 ; j < n ; i++) { v++ } }; return v }
Example: nested loop.

Calculations

Big O calculations can be combined:

Consider multiple, nested, loops: O(x) * O(y) = O(x*y)
Consider multiple, un-nested, loops: O(x) + O(x) = O(2x)
Consider a linear loop and a logarithmic, nested, loop: O(x) * O(log(y)) = O(x*log(y))

Algorithms: Space Complexity

There’s a lot more ambiguity and disagreement about what constitutes what with respect to Space Complexity: https://www.studytonight.com/data-structures/space-complexity-of-algorithms observes that:

“Sometimes Auxiliary Space is confused with Space Complexity. But Auxiliary Space is the extra space or the temporary space used by the algorithm during it's execution.”

Furthermore, small input sizes are often just called O(1) (despite being strictly linear): https://stackoverflow.com/questions/44430974/space-complexity-of-an-array.

Generally, carefully explaining how the memory footprint changes (or doesn’t) based on various input sizes is sufficient (regardless of the label we apply in some ambiguous outlier cases).
Arguably, in such cases one can use either moniker or label (linear or constant) depending on one’s justification. In other words, the labels above might overlap in certain limited scenarios rather than being sharply bounded/mutually exclusive.

Big O Notation

Is similar to Big O Notation for Time Complexity.

O(1)
- Space in memory is not dependent on inputs.
- Space in memory remains fixed in size.
O(n)
- Space in memory is proportional (linearly-related) to input size. For very small sizes this is often considered O(1).
O(log(n))
- Space in memory gradually decreases through the runtime of an instruction set, method, or operation.
O(nˣ)
- Space in memory increases exponentially relative to input size.

Algorithms: Famous Algorithms

Minimum Spanning Tree

Kruskal’s Algorithm

Greedy algorithm
Time Complexity: O(V²) | V is the number of vertices
Given an undirected, connected, weighted Graph, construct a minimum spanning Tree of it.
- Traverse an entire Graph using the minimum number of moves and weights.
- Applies to undirected, connected, weighted Graphs.
Iterate through each node of the Graph sequentially starting with the lowest node (or first sorted node):
- Find the minimum weighted path to another node such that no cycle is formed.
- Each starting node is tracked as its own Tree.
- If a cycle is formed, that Tree is discarded.
- The remaining Tree is the minimum spanning Tree.
Review: https://www.geeksforgeeks.org/kruskals-minimum-spanning-tree-algorithm-greedy-algo-2/

Prim’s Algorithm

Greedy algorithm
Time Complexity: O(E*log(V)) | E is the number of edges and V is the number of vertices
Prim’s algorithm uses two sets of vertices to supplement its operation - visited and not yet visited nodes.
Constructs a minimum spanning Tree from a Graph.
- Traverse an entire Graph using the minimum number of moves and weights.
- Applies to undirected, connected, weighted Graphs.
Kruskal’s Algorithm is similar to Prim’s Algorithm which uses a single Tree (whereas Kruskal’s Algorithm uses a metaphorical forest).
Prim’s Algorithm traverses each node more than once whereas Kruskal’s Algorithm allows for each node to be traversed only once (within the last remaining, valid, Tree).
Review: https://www.coursera.org/learn/cs-fundamentals-3/home/week/4

Dijkstra's Algorithm

Find the shortest path between nodes in a graph.
- Find the minimum weighted path from the current or starting node to any node it’s directly connected to or any path not yet visited from a previously visited node.
  - Mark the node that’s the minimum weight from the current or starting node as visited.
  - Set the newly visited node as the new current node.
  - Set any previously unvisited nodes that are directly connected to any previously visited nodes as visited.
- Repeat until the endpoint has been reached.

Heap’s Algorithm

Used to find all permutations of a set.
Recursively moves index and swaps pairs in transfer array.
- https://leetcode.com/problems/permutations/

// Heap's Algorithm
// https://stackoverflow.com/questions/27539223/permutations-via-heaps-algorithm-with-a-mystery-comma#27540053
function permutation(originalArr) {
  let transferArr = originalArr, resultsAsStrings = []

  function swap(end, begin) {
    let original = transferArr[begin]
    transferArr[begin] = transferArr[end]
    transferArr[end] = original
  }

  function heaps_algorithm(n) {
    if (n === 1) resultsAsStrings.push(transferArr.join(""))

    for (let i = 0; i != n; ++i) {
      heaps_algorithm(n - 1)
      swap(n - 1, n % 2 === 0 ? i : 0)
    }
  }

  heaps_algorithm(originalArr.length)
  return resultsAsStrings
}

const a = [1, 2, 3, 4]
console.log(permutation(a))

const b = [0, 1, 2]
console.log(permutation(b))

const c = [2, 3, 4]
console.log(permutation(c))

Algorithms: Search

A Needle refers to the Value, item, or Object one is searching for within a Haystack (the collection of items one is searching within).

Search Techniques

Breadth First Search (BFS) - graph search, sequential horizontal search (by row or width) of a node.

Typically uses Queue.
Pros: Will find a Needle and the shortest path in unweighted graphs.
Cons: If the Needle is far from starting point, use DFS.

Depth First Search (DFS) - graph search, sequential vertical search (by column or full height) of a branch.

Typically uses Stack.
Pros: Will find a Needle (if no infinite cycles exist).
Cons: Not guaranteed to find the shortest path. If the Needle is close to the source, use BFS. Susceptible to infinite cycles if such exist within the Haystack.

Sorted by Average Big O Time Complexity.

Hash Table - define a hashing method, each value is assigned a hash key that is unique to that value, then lookup a value using the hash key.

Best: O(1)
Average: O(1)
Worst: O(n)

const hash = (needle, haystack) => {
  console.log((`Find: ${needle} in [${haystack}]`, "hashstart")

  let table = []
  for (let i = 0; i < haystack.length; i++) {
    table[i] = -1
  }

  let keyHash = 0

  const simpleHash = val => val % table.length

  for (let i = 0; i < haystack.length; i++) {
    let flag = haystack[i] == needle
    const currentHash = simpleHash(haystack[i])

    if (table[currentHash] === -1) {
      table[currentHash] = haystack[i]
      if (flag) keyHash = currentHash

    } else {
      const NEXT = table.indexOf(-1)
      table[NEXT] = haystack[i]
      if (flag) keyHash = NEXT

    }
  }

  console.log((`Hash Table: [${table}] - Length: ${table.length}`)

  const BEGIN = new Date()
  const result = table[keyHash]
  const indexOf = haystack.indexOf(needle)
  const END = new Date()

  console.log((`Hash: ${keyHash} - Value: ${result} - IndexOf ${indexOf} in unsorted haystack`)
  console.log(`Time taken: ${END - BEGIN}`)
}

Linear - iterate through n until k is found.

Best: O(1)
Average: O(n)
Worst: O(n)

const linear = (needle, haystack) => {
  const BEGIN = new Date()
  let result = -1

  console.log(`Find: ${needle} in [${haystack}]`)

  for (let i = 0; i < haystack.length; i++) {
    if (needle === haystack[i]) {
      result = i;
      break;
    }
  }

  const indexOf = haystack.indexOf(needle);
  const END = new Date()
  console.log(`Index: ${result} - IndexOf: ${indexOf}`)
  console.log(`Time taken: ${END - BEGIN}`)
}

Binary- take a sorted Array and recursively divide in two comparing the Needle against values in each divided Array.

Best: O(1)
Average: O(log(n))
Worst: O(log(n))

const binary = (needle, haystack) => {
  haystack.sort((a, b) => a - b)
  console.log(`Sorted haystack: ${haystack}`)

  let l = 0, r = haystack.length - 1

  while (l <= r) {
    const M = Math.floor((l + r) / 2)

    if (haystack[M] === needle) {
      console.log(`${haystack[M]} at sorted index ${M}`)
      break
    } 

    if (haystack[M] < needle) l = M + 1

    // haystack[M] > needle
    else r = M - 1
  }
}

Consult a clean implementation here: https://www.instagram.com/reel/CYjmjbFF_8g/?utm_source=ig_web_copy_link

Resources and Links

https://www.instagram.com/reel/CYjmjbFF_8g/?utm_source=ig_web_copy_link

Algorithms: Sort

Sorted by Average Big O Time Complexity.

Non-Comparison Sorts

Insertion Sort - use a second array to rebuild the first array in sorted order, loop through inserting the next lowest value into the second array.

Best: O(n)
Average: O(n²)
Worst: O(n²)

// Two arrays - lowest from original goes into next open spot in result
const insertion = arr => {
  console.log(`Starting array: ${arr} - Length: ${arr.length}`)
  const BEGIN = new Date()

  /** Algorithm begins */
  const inner = (arr, result = []) => {
    let lowest = arr[0], lowestIndex = 0
    const L = arr.length

    for (let i = 0; i < L; i++) {
      if (arr[i] < lowest) {
        lowest = arr[i]
        lowestIndex = i
      }
    }

    result.push(lowest)

    if (lowestIndex === 0) arr = arr.slice(lowestIndex, L)

    if (lowestIndex === L - 1) arr = arr.slice(0, lowestIndex)

    if (lowestIndex < L) {
      const first = arr.slice(0, lowestIndex)
      const second = arr.slice(lowestIndex + 1, L)
      arr = first.concat(second)
    }

    if (L > 0) return inner(arr, result)
    else return result
  }
  let result = inner(arr)
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${result} - Length: ${result.length} - Time: ${END - BEGIN}`)
}

Counting Sort - sort by statistical frequency using a buffer. (Frequency Sort)

Best: O(n + k)
Average: O(n + k)
Worst: O(n + k)

// Sort by frequency
const bucket = arr => {
  let freq = [], result = [], max = arr[0]

  for (let i = 0; i < arr.length; i++) {
    if (max < arr[i]) max = arr[i]
  }

  for (let i = 0; i < max + 1; i++) {
    freq[i] = 0
  }

  console.log(`Starting array: ${arr} - Length: ${arr.length}`)
  const BEGIN = new Date()

  /** Algorithm begins */
  for (let i = 0; i < arr.length; i++) {
    freq[arr[i]] = freq[arr[i]] + 1
  }

  console.log(`Frequency array: ${freq}`, "bucketfreq")
  for (let i = 0; i < freq.length; i++) {
    for (let j = 0; j < freq[i]; j++) {
      result.push(i)
    }
  }
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${result} - Length: ${result.length} - Time: ${END - BEGIN}`)
}

Bucket Sort- sort an array using buckets (that divide across the range of the input array), can be a Counting Sort and/or Insertion Sort depending on setup.

Best: O(n + k)
Average: O(n²)
Worst: O(n²)

Comparison Sorts

Heap Sort - properly sorts an array into a Binary Heap.

Best: O(n * log(n))
Average: O(n * log(n))
Worst: O(n * log(n))

Review: https://github.com/Thoughtscript/cplusplus-coursera/blob/master/course/12%20-%20Heap/main.cpp

Merge Sort - divide an array into halves until two elements exist in each array then recombine comparing elements along the way.

Best: O(n * log(n))
Average: O(n * log(n))
Worst: O(n * log(n))

// Divide an array into halves until two elements exist in each array then recombine comparing elements along the way
const mergesort = arr => {
  const BEGIN = new Date()
  console.log(`Starting array: ${arr} - Length: ${arr.length}`)

  /** Algorithm begin */
  const merge = (a, b) => {
    let i = 0, j = 0, results = []

    while (i < a.length && j < b.length) {
      if (a[i] < b[j]) {
        results.push(a[i])
        i++

      } else {
        results.push(b[j])
        j++
      }
    }

    while (i < a.length) {
      results.push(a[i])
      i++
    }

    while (j < b.length) {
      results.push(b[j])
      j++
    }

    return results
  }

  const innerMerge = arr => {
    if (arr.length <= 1) return arr
    const MID = Math.floor(arr.length / 2)
    const LEFT = innerMerge(arr.slice(0, MID))
    const RIGHT = innerMerge(arr.slice(MID))
    return merge(LEFT, RIGHT)
  }

  const result = innerMerge(arr)
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${result} - Length: ${result.length} - Time: ${END - BEGIN}`)
}

Review: https://medium.com/analytics-vidhya/implement-merge-sort-algorithm-in-javascript-7402b7271887

Quick Sort - specify a pivot point and swap values that are above and below the pivot point.

Best: O(n * log(n))
Average: O(n * log(n))
Worst: O(n²)

// Specify a pivot point and swap
const quicksort = arr => {
  let result = []
  console.log(`Starting array: ${arr} - Length: ${arr.length}`)
  const BEGIN = new Date()

  /** Algorithm begin */
  const inner = (arr, pivot, pointer) => {
    const L = arr.length
    let rArr = [], lArr = [], flag = false

    for (let i = 0; i < L; i++) {
      if (arr[i] > arr[pivot]) rArr.push(arr[i])
      else lArr.push(arr[i])

      const rl = rArr.length, ll = lArr.length
      if (!flag && (ll > 1 || rl > 1)) {
        if (lArr[ll - 1] < lArr[ll - 2]) flag = true
        if (rArr[rl - 1] < rArr[rl - 2]) flag = true
      }
    }

    arr = lArr.concat(rArr)
    pointer++
    if (pointer > L - 1 || !flag) return arr
    else {
      let right = pointer < Math.floor(L / 2) + 1 ? inner(arr, L - pointer, pointer) : arr
      return Math.floor(pivot / 2) > 1 ? inner(right, Math.floor(pivot / 2), pointer) : arr
    }
  }
  result = inner(arr, arr.length - 1, 0)
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${result} - Length: ${result.length} - Time: ${END - BEGIN}`)
}

Bubble Sort - loop through replacing sequential pairs (Swap) as necessary.

Best: O(n)
Average: O(n²)
Worst: O(n²)

// Loop through replacing sequential pairs as needed
const bubble = arr => {
  console.log(`Starting array: ${arr} - Length: ${arr.length}`)
  const BEGIN = new Date()

  /** Algorithm begins */
  for (let i = 0; i < arr.length - 1; i) {
    if (arr[i] > arr[i + 1]) {
      const original = arr[i]
      arr[i] = arr[i + 1]
      arr[i + 1] = original
      i = 0
    } else i++
  }
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${arr} - Length: ${arr.length} - Time: ${END - BEGIN}`)
}

Selection Sort - loop through swapping the lowest number with the current index.

Best: O(n²)
Average: O(n²)
Worst: O(n²)

// Loop through swapping the lowest number
const selection = arr => {
  console.log(`Starting array: ${arr} - Length: ${arr.length}`)
  const BEGIN = new Date()

  /** Algorithm begins */
  const inner = (arr, lastIndex = 0) => {
    let lowest = arr[lastIndex], lowestIndex = lastIndex, flag = false
    for (let i = lastIndex; i < arr.length; i++) {
      if (arr[i] <= lowest) {
        lowest = arr[i]
        lowestIndex = i
        flag = true
      }
    }

    if (flag) {
      const original = arr[lastIndex]
      arr[lastIndex] = lowest
      arr[lowestIndex] = original
      lastIndex++
      return inner(arr, lastIndex)
    } else return arr
  }

  let result = inner(arr)
  /** Algorithm ends */

  const END = new Date()
  console.log(`Ending array: ${result} - Length: ${result.length} - Time: ${END - BEGIN}`)
}

Language Implementations

Java uses a modified Merge Sort (by default).
JavaScript uses a modified Quick Sort with Insertion Sort fallback (by default).

Resources and Links

Algorithms: Recursion

Where a function repeatedly calls itself until some condition is met.

Solved Examples

Algorithms: Loop Recursion

A kind of Recursion problem where Recursion is called within a For-loop.

Useful for iterating through all permissible contiguous String subsequence "chunks".

Resources and Links

https://www.youtube.com/watch?v=DI6pH9Cx654 <- great example

Algorithms: Dynamic Programming

Breaking a problem down into multiple sub-problems where each step is stored (typically on the fly not Ahead-of-Time).

Often used with Memoization to reduce recursion footprint.

Solved Examples

https://leetcode.com/problems/count-sorted-vowel-strings/ <- Using variables that are saved off each cycle
https://leetcode.com/problems/product-of-array-except-self/ <- Bottom 50% time-complexity. Top .07% space-complexity.
https://leetcode.com/problems/max-increase-to-keep-city-skyline/
https://leetcode.com/problems/arithmetic-subarrays/
https://leetcode.com/problems/pascals-triangle/

Algorithms: Memoization

Saving the state of an instruction set into an Array, Map, Stack, Dequeue, or Queue.

Often eliminates the need to recursively compute some value.

Solved Examples

https://leetcode.com/problems/triangle/
https://leetcode.com/problems/climbing-stairs/ <- Used a precomputed O(1) data set I generated.
https://leetcode.com/problems/count-and-say/ <- Used a precomputed O(1) data set I generated.
https://leetcode.com/problems/longest-nice-substring/

Algorithms: Sets

Power Set

Dynamic Solution:

/**
 * Review: https://stackoverflow.com/questions/24365954/how-to-generate-a-power-set-of-a-given-set
 *
 * Strategy is clean and elegant so I made a few tweaks and implemented it in JS from the above.
 * 
 * Add elements one at a time to a result array, also iterate over the result array adding elements to the existing ones.
 *
 * This accomplished what was done here: https://medium.com/leetcode-patterns/leetcode-pattern-3-backtracking-5d9e5a03dc26 
 * and here: https://jimmy-shen.medium.com/78-subsets-fa4f0b047664 without backtracking (those are excellent articles 
 * and implementations given the requirements!).
 */

const powerSet = arrSet => {
  let arrSets = []
  for (let i = 0; i < arrSet.length; i++) {
    let item = arrSet[i], origSets = [...arrSets]

    for (let j = 0; j < origSets.length; j++) {
      let nSet = origSets[j].concat(item)
      arrSets.push(nSet)
    }
    arrSets.push([item])
  }
  arrSets.push([])
  return arrSets.sort()
}

window.onload = () => {
  console.log(powerSet([4, 2, 3]))
  console.log(powerSet([5, 1, 4, 2, 3]))
}

[
  [],          [ 2 ],
  [ 2, 3 ],    [ 3 ],
  [ 4 ],       [ 4, 2 ],
  [ 4, 2, 3 ], [ 4, 3 ]
]

[
  [],             [ 1 ],
  [ 1, 2 ],       [ 1, 2, 3 ],
  [ 1, 3 ],       [ 1, 4 ],
  [ 1, 4, 2 ],    [ 1, 4, 2, 3 ],
  [ 1, 4, 3 ],    [ 2 ],
  [ 2, 3 ],       [ 3 ],
  [ 4 ],          [ 4, 2 ],
  [ 4, 2, 3 ],    [ 4, 3 ],
  [ 5 ],          [ 5, 1 ],
  [ 5, 1, 2 ],    [ 5, 1, 2, 3 ],
  [ 5, 1, 3 ],    [ 5, 1, 4 ],
  [ 5, 1, 4, 2 ], [ 5, 1, 4, 2, 3 ],
  [ 5, 1, 4, 3 ], [ 5, 2 ],
  [ 5, 2, 3 ],    [ 5, 3 ],
  [ 5, 4 ],       [ 5, 4, 2 ],
  [ 5, 4, 2, 3 ], [ 5, 4, 3 ]
]

Greedy Solution:

/**
 * One can approach this from a few different angles:
 * 1. Brute force.
 * 2. Binary counter.
 * 3. Use native abstractions.
 * ---
 * I'd prefer to use what I understand to be a novel approach by generating 
 * constraints AOT (ahead-of_time) so that power-set 
 * generation is linear for every subsequent run.
 * ---
 * We can then use these values to populate others by index (e.g. - 1 means in the subset, 0 not).
 * The upside is that this is superfast (most powerset implementations 
 * involve deep recursion and 2-3 nested loops).
 * ----
 * The downside of this approach is that only works if you know the desired 
 * set size beforehand. It's Greedy.
 */

const generateConstraints = () => {
  let result = [], current = []

  // Flattened nested loop...
  for (let i = 0, j = 0, k = 0, l = 0; i < 2 && j < 2 && k < 2 && l < 2; i) {
    current.push(i)
    current.push(j)
    current.push(k)
    current.push(l)
    result.push(current)
    current = []

    l++;
    if (l == 2) {
      k++
      l = 0
    }
    if (k == 2) {
      j++
      k = 0
    }
    if (j == 2) {
      i++
      j = 0
    }
  }

  return result
}

window.onload = () => {
  const aot = generateConstraints()
  console.log(aot)
}

[
  [],          [ 2 ],
  [ 2, 3 ],    [ 3 ],
  [ 4 ],       [ 4, 2 ],
  [ 4, 2, 3 ], [ 4, 3 ]
]

[
  [],             [ 1 ],
  [ 1, 2 ],       [ 1, 2, 3 ],
  [ 1, 3 ],       [ 1, 4 ],
  [ 1, 4, 2 ],    [ 1, 4, 2, 3 ],
  [ 1, 4, 3 ],    [ 2 ],
  [ 2, 3 ],       [ 3 ],
  [ 4 ],          [ 4, 2 ],
  [ 4, 2, 3 ],    [ 4, 3 ],
  [ 5 ],          [ 5, 1 ],
  [ 5, 1, 2 ],    [ 5, 1, 2, 3 ],
  [ 5, 1, 3 ],    [ 5, 1, 4 ],
  [ 5, 1, 4, 2 ], [ 5, 1, 4, 2, 3 ],
  [ 5, 1, 4, 3 ], [ 5, 2 ],
  [ 5, 2, 3 ],    [ 5, 3 ],
  [ 5, 4 ],       [ 5, 4, 2 ],
  [ 5, 4, 2, 3 ], [ 5, 4, 3 ]
]

Cartesian Multiplication

Original Generalized Dynamic Solution using the above Dynamic Power Set Solution as an inspiration:

const solve = (args) => {
    let last = []
    const L = args.length - 1

    for (let i = 0; i < args[L].length; i++) {
        last.push([args[L][i]])
    }

    for (let i = L - 1; i >= 0; i--) {
        let curr = args[i], temp = []

        for (let j = 0; j < curr.length; j++) {
            for (let k = 0; k < last.length; k++) {
                const v = [...last[k]]
                if (v.indexOf(curr[j]) === -1) v.push(curr[j])
                temp.push(v)
            }
        }

        last = temp
    }

    console.log(last)
} 

window.onload = () => {
  const argsA = [[1,2],[3,4],[5,6]]
  solve(argsA)

  const argsB = [[1,2,3],[3,4],[5,6,15]]
  solve(argsB)

  const argsC = [[1],[1],[1]]
  solve(argsC)
}

Note: the above implementation isn't order preserving (Sets don't preserve order anyway) and assumes no empty Array is passed (the Empty Set is assumed to be present in every Set so we'll just omit all empty Arrays/Empty Set or we would have to add it everywhere).

[
  [5, 3, 1], 
  [6, 3, 1], 
  [5, 4, 1], 
  [6, 4, 1], 
  [5, 3, 2], 
  [6, 3, 2], 
  [5, 4, 2], 
  [6, 4, 2]
]

[
  [5, 3, 1], 
  [6, 3, 1],
  [15, 3, 1], 
  [5, 4, 1], 
  [6, 4, 1], 
  [15, 4, 1], 
  [5, 3, 2], 
  [6, 3, 2], 
  [15, 3, 2], 
  [5, 4, 2], 
  [6, 4, 2], 
  [15, 4, 2],
  [5, 3], 
  [6, 3],
  [15, 3], 
  [5, 4, 3], 
  [6, 4, 3], 
  [15, 4, 3]
]

[
  [1]
]

Algorithms: Hash Maps and Counts

Typically used when a solution requires a unique answer or a deduplicated Set.

Also often used to track the frequency of some value (number of occurrences).

Technique: Hash Map

let M = {}; 

// ...

if (M[x] != undefined) M[x]++; 
else M[x] = 1;

Solved Examples

Algorithms: Graph Relationship

Where a series of relationships are expressed using Arrays, Sets, or Maps.

Often a kind of Hash Counting Algorithm that requires Reverse Maps, Dual Mapping, Symmetric Maps and the like. Strong possibility that a solution can be achieved using a Maps of Maps.

Reverse Map

const A = {
    "AA": "BB",
    "CC":"DD"
}

const B = {
    "BB": "AA",
    "DD":"CC"
}

N-Tuple

Demonstrating a specific kind of this problem and how to index the relationships in multiple ways in O(N) time:

const Example = [
    [1, 2, 3], 
    [0, 3, 1], 
    [0, 4, 3], 
    [0, 5, 3], 
]

/*
 Where for each X in Example:
    1. X[1] is related to X[2] through relationship X[0].
    2. Where the relationship of X[1] to X[2] is symmetric (or not depending).
*/

const A = {
    "1": {
        "3": 0
    },
    "2": {
        "3": 1
    },
    "3": {
        "2": 1,
        "1": 0,
        "4": 0,
        "5": 0
    },
    "4": {
        "3": 0
    },
    "5": {
        "3": 0
    }
}
// Map of X[1] to X[2] and X[2] to X[1]

const B = {
    "1": {
        "3": {
            "2": true
        }
        "2": {
            "3": true
        }
    },
    "0": {
        "1": {
            "3": true
        },
        "3": {
            "1": true,
            "4": true,
            "5": true
        },
        "4": {
            "3": true
        },
        "5": {
            "3": true
        }
    }
}

// Map of X[0] to the relationship of X[2] to X[1] (and vice-versa)

Algorithms: Phone Dialer

Questions that use a 0-9 phone pad or phone numbers.

Solved Examples

Algorithms: Text Representation

Technique: Unicode Char Codes

// JavaScript
const c = str.charCodeAt(i);

// Unicode numbers codes 48-57 inclusive
if (c >= 48 && c <= 57) console.log("I'm a number character")
// Unicode uppercased letters codes 65-90 inclusive 
if (c >= 65 && c <= 90) console.log("I'm an uppercased character")
// Unicode lowercased letters codes 97-122 inclusive 
if (c >= 97 && c <= 122) console.log("I'm an lowercased character")

// JavaScript
const c = str.charCodeAt(i);

if ((c >= 65 && c <= 90) || (c >= 97 && c <= 122)) {
    const isUpperVowel = [65, 69, 73, 79, 85].indexOf(c) === -1 
    const isLowerVowel = [97, 101, 105, 111, 117].indexOf(c) === -1
    if (isUpperVowel && isLowerVowel) console.log("I'm a consonant")
    else console.log("I'm a vowel")
} else console.log("I'm not a letter")

Review: https://tutorial.eyehunts.com/js/get-the-unicode-value-of-character-javascript-example-code/

Technique: Character Maps

Alphabet maps:

// JavaScript
const a = {"a":0,"e":0,"i":0,"o":0,"u":0,"b":0,"c":0,"d":0,"f":0,"g":0,"h":0,"j":0,"k":0,"l":0,"m":0,"n":0,"p":0,"q":0,"r":0,"s":0,"t":0,"v":0,"w":0,"x":0,"y":0,"z":0}
const A = {"A":0,"E":0,"I":0,"O":0,"U":0,"B":0,"C":0,"D":0,"F":0,"G":0,"H":0,"J":0,"K":0,"L":0,"M":0,"N":0,"P":0,"Q":0,"R":0,"S":0,"T":0,"V":0,"W":0,"X":0,"Y":0,"Z":0}

// As array mapped by index
const arr_a = Object.keys(a)
const arr_A = Object.keys(A)

Consonant maps:

// JavaScript
const c = {"b":0,"c":0,"d":0,"f":0,"g":0,"h":0,"j":0,"k":0,"l":0,"m":0,"n":0,"p":0,"q":0,"r":0,"s":0,"t":0,"v":0,"w":0,"x":0,"y":0,"z":0}
const C = {"B":0,"C":0,"D":0,"F":0,"G":0,"H":0,"J":0,"K":0,"L":0,"M":0,"N":0,"P":0,"Q":0,"R":0,"S":0,"T":0,"V":0,"W":0,"X":0,"Y":0,"Z":0}

// As array mapped by index
const arr_c = Object.keys(c)
const arr_C = Object.keys(C)

Vowel maps:

// JavaScript
const v = {"a":0,"e":0,"i":0,"o":0,"u":0}
const V = {"A":0,"E":0,"I":0,"O":0,"U":0}

// As array mapped by index
const arr_v = Object.keys(v)
const arr_V = Object.keys(V)

Number maps:

// JavaScript
const N = {"0":0,"1":0,"2":0,"3":0,"4":0,"5":0,"6":0,"7":0,"8":0,"9":0}

// As array mapped by index
const arr_N = Object.keys(N)

Technique: Check Character is Number

// Java
Character.isDigit(inputStr.charAt(i)); // true/false

Algorithms: Letter Chars and Word Dictionaries

Specific to the dictionary, String, and Character checking with a fixed, finite, alphabet.

Can often be simplified to iterating through the letters of the English alphabet (0-25) rather than iterating over a string or multiple nested loops.
Otherwise, Hashing is often a good approach since the number of map keys will often be somewhere between 0-52 (inclusive).

Solved Examples

https://leetcode.com/problems/expressive-words/
https://leetcode.com/problems/string-without-aaa-or-bbb/ <- Bottom 50% time-complexity.
https://leetcode.com/problems/maximum-number-of-words-you-can-type/
https://leetcode.com/problems/check-if-all-characters-have-equal-number-of-occurrences/ <- Bottom 50% time-complexity.
https://leetcode.com/problems/check-if-word-equals-summation-of-two-words/
https://leetcode.com/problems/long-pressed-name/
https://leetcode.com/problems/shortest-word-distance/
https://leetcode.com/problems/keyboard-row/
https://leetcode.com/problems/verifying-an-alien-dictionary/ <- Bottom 50% time-complexity.
https://leetcode.com/problems/single-row-keyboard/
https://leetcode.com/problems/remove-vowels-from-a-string/ <- Bottom 50% time-complexity.

Algorithms: Word Patterns, Anagrams, and Palindromes

Word pattern, anagram, and palindrome scenarios.

Solved Examples

https://leetcode.com/problems/word-pattern/
https://leetcode.com/problems/palindrome-number/
https://leetcode.com/problems/valid-palindrome/ <- Below 50% time-complexity. Top 2.62% space-complexity.
https://leetcode.com/problems/palindrome-linked-list/
https://leetcode.com/problems/valid-palindrome-ii/
https://leetcode.com/problems/palindromic-substrings/ <- Below 50% time-complexity. Brute-force.
https://leetcode.com/problems/partition-labels/ <- Below 50% time-complexity.
https://leetcode.com/problems/find-anagram-mappings/
https://leetcode.com/problems/group-anagrams/ <- Below 50% time-complexity. Top .26% space-complexity.
https://leetcode.com/problems/minimum-number-of-steps-to-make-two-strings-anagram/

Algorithms: Parentheses

Calculate the validity of various Strings or substrings containing parentheticals.

Similar to facing but typically with a narrower focus (balance).

Somewhat similar to left, right two-pointer problems

Solved Examples

Algorithms: Two Pointer

Typically used when working with Arrays or Strings.

Use two pointers left and right.

Check or compare the values of each cycle.

Technique: Left and Right Pointers

while (left < right) { 
    //... 
    left++; 
    right--;
}

Solved Examples

Algorithms: Sliding Window

Often used to find unique subarrays, longest substrings, longest Strings with some property, etc.

This method is often used to reduce the time complexity of a solution from Quadratic to Linear time (when it can be applied).

Generally:

Initialize two pointers (left and right) at the beginning.
Expand the window (by incrementing the right pointer) until some desired condition is met.
Then, contract the window by incrementing the left pointer

Solved Examples

Algorithms: Direction and Facing

Used to represent cardinal direction facing.
Use a common local variable to track the current index allowing the index to move from end to start or start to end.
- Involves mod or Modulus concepts.
- Once the index reaches a certain point, it wraps around like a ring or circle.
Use an Array (usually for three or more values) or pointers to represent possible moves
- e.g. [n, e, s, w] or [up, right, down, left]

Solved Examples

Algorithms: Number Representation

Non-number type number-representation scenarios.

Technique: Fast Number Reversal

Old way:

const rev = num => {
    const numStr = `${num}`
    let str = ''

    for (let i = numStr.length - 1; i >= 0; i--) {
        str += numStr[i]
    }

    return parseInt(str)
}

Much faster than int to String and back conversion:

// Java
int reverse(int num) {
    int rev = 0;

    while(num > 0){
        int d = num % 10;
        rev = rev * 10 + d;
        num = num / 10;
    }

    return rev;
}

// Java
const reverse = num => {
    let rev = 0

    while(Math.floor(num) > 0) {
        const d = num % 10
        result = Math.floor(result * 10 + d)
        num = num / 10
    }

    return rev
}

JavaScript doesn't automatically round Numbers < 1 to 0 like Java does. So, use Math.floor().

Solved Examples

https://projecteuler.net/problem=89
https://www.codewars.com/kata/51b66044bce5799a7f000003
https://www.codewars.com/kata/5324945e2ece5e1f32000370
https://www.codewars.com/kata/525f4206b73515bffb000b21
https://www.codewars.com/kata/5265326f5fda8eb1160004c8
https://www.codewars.com/kata/54d7660d2daf68c619000d95
https://leetcode.com/problems/thousand-separator/
https://leetcode.com/problems/roman-to-integer/ <- Below 50% time-complexity. Top .01% space-complexity.
https://leetcode.com/problems/integer-to-roman/
https://leetcode.com/problems/integer-to-english-words/
https://leetcode.com/problems/string-to-integer-atoi/
https://leetcode.com/problems/add-strings/
https://leetcode.com/problems/add-two-numbers/
https://leetcode.com/problems/add-two-numbers-ii/ <- Below 50% time-complexity. Top 2.73% space-complexity.
https://leetcode.com/problems/sum-of-two-integers/
https://leetcode.com/problems/excel-sheet-column-number/ <- Not original, bijective base 26.

Algorithms: Number Theory

Implementations of arithmetic or number-theoretic scenarios.

These may or may not have some trick to them (like Big Int addition).
Factorials.

Solved Examples

Algorithms: Prime Numbers

Algorithms involving Prime Numbers.

Solved Examples

Algorithms: Path Compression and Merging

Used to combine multiple subarrays until they are all non-overlapping.

Used to merge Sets until they are Disjoint Sets.

Solved Examples

https://leetcode.com/problems/sum-of-digits-of-string-after-convert/ <- top 98.6% answer, loosely a compression/merging problem since it involves repeatedly summing a number until it’s below 10 or k.
https://leetcode.com/problems/merge-intervals/
https://www.codewars.com/kata/5286d92ec6b5a9045c000087
https://leetcode.com/problems/partition-labels/
https://leetcode.com/problems/meeting-rooms/
https://leetcode.com/problems/meeting-rooms-ii/ <- Very slow.
https://www.codewars.com/kata/52b7ed099cdc285c300001cd
https://leetcode.com/problems/summary-ranges/

Algorithms: Directory or Name Traversal

Directory name, IP Address, versioning, or URL context path operations.

Solved Examples

Algorithms: Level Recursion

Recursion specific to iterating down a pyramidal data structure.

Where each following level has a number of elements greater than or equal to the prior one.
Or, just when there are levels, period.

Technique: Queue

Use a Queue (nxt), use two alternating containers: level and nxt.

    level = [node]
    nxt = []

    while len(level) > 0:
        temp = []
        count = 0
        l = len(level)

        while count < l:
            if (len(level) > 0):
                c = level.pop(0)
                if c is None:
                    continue
                temp.append(c.value)
                nxt.append(c.left)
                nxt.append(c.right)
            count += 1

        if len(temp) > 0:
            for x in range(0, len(temp)):
                result.append(temp[x])

        level = nxt

Solved Examples

Algorithms: Geometry

Implementations of geometry scenarios and problems.

Solved Examples

Algorithms: Islands

Problems representing connected sub-matrices.

Technique: Check and Recurse

const countIslands = mapStr => {
  const ARR = mapStr.split("\n"), cleanedArr = []

  for (let i = 0; i < ARR.length; i++) {
    cleanedArr.push(ARR[i].split(""))
  }

  let deepCopy = [...cleanedArr], cnt = 0, hasNext = findNext(deepCopy)

  while (hasNext !== false) {
    cnt++
    recurse(deepCopy, hasNext[0], hasNext[1])
    hasNext = findNext(deepCopy)
  }

  return cnt
}

const findNext = arr => {
  for (let i = 0; i < arr.length; i++) {
    for (let j = 0; j < arr[i].length; j++) {
      const I = arr[i][j]
      if (I === '0') {
        return [i, j]
      }
    }
  }  
  return false
}

const recurse = (arr, i, j) => {
  if (i >= arr.length || j >= arr[i].length || i <= -1 || j <= -1) {}
  else {
    arr[i][j] = '.'
    if (arr[i-1] !== undefined && arr[i-1][j] === '0') recurse(arr, i-1, j)    
    if (arr[i+1] !== undefined && arr[i+1][j] === '0') recurse(arr, i+1, j)
    if (arr[i][j-1] !== undefined && arr[i][j-1] === '0') recurse(arr, i, j-1)    
    if (arr[i][j+1] !== undefined && arr[i][j+1] === '0') recurse(arr, i, j+1)
  }
}

Solved Examples

Algorithms: Pathing

To find a path from some origin to some end point.

Solved Examples

Algorithms: Rotations, Spirals, Diagonals

Includes spirals or diagonally traversing an N x M Array or matrix, transposing matrices, rotations, etc.

Also often involves mod or Modulus concepts.

Used for ciphers or rotating an Array.

Solved Examples

Algorithms: Coins and Make Bricks

Optimization scenarios involving fixed units of some value that need to be combined in some optimal way.

Solved Examples

https://projecteuler.net/problem=31
https://www.codewars.com/kata/564d0490e96393fc5c000029
http://www.javaproblems.com/2013/11/java-logic-2-makebricks-codingbat.html
https://leetcode.com/problems/coin-change-ii/ <- Not original.
https://leetcode.com/problems/coin-change/ <- Not original. Slow.
https://leetcode.com/problems/how-many-apples-can-you-put-into-the-basket/
https://leetcode.com/problems/water-bottles/
https://leetcode.com/problems/maximum-units-on-a-truck/ <- Below 50% time-complexity.

Algorithms: N-Sum

Triplet, 4-Sum, 3-Sum, and 2-Sum type problems.

Solved using a mix of l,r pointer and hashmap techniques.

Solved Examples

https://leetcode.com/problems/two-sum/ <- Original solution was very slow but top 100% space-complexity.
https://leetcode.com/problems/two-sum-less-than-k/
https://leetcode.com/problems/two-sum-ii-input-array-is-sorted/

Algorithms: Mountains, Peaks, and Stock Markets

Find a highest point in a sequence and the next highest or lowest point depending.

Solved Examples

Algorithms: Controlled Subsequences

Track things like substrings of a certain length, row or text justification/formatting, or repeated sub-patterns.

Use multiple pointers to track some value that gets reset every n-many loops.

Also includes "reverse loop in loop" (loop "backfilling") problems.

Solved Examples

Data Structures: Trees

A "tree-like", derived, data structure that has zero or more child instances, each instance taking a value.

Attributes

Value
children - empty or other Tree Node instances.

Implementations

var TreeNode = function(val, children = []) {
    this.val = val;
    this.children = children;
}

Depth First Search

Refer to the Search article.

Breadth First Search

Level Recursion - consult the Level Recursion article.

Notes

A LinkedList can be thought of as Tree with only one child element at each node.

Data Structures: Binary Trees

A "tree-like", derived, data structure that connects left and right child instances, each instance taking a value.

Attributes

Value
left - null, nil, or none or another Binary Tree Node instance.
right - null, nil, or none or another Binary Tree Node instance.

Implementations

var TreeNode = function(val, left, right) {
    this.val = val;
    this.left = left;
    this.right = right;
}

Depth First Search

Pre Order - current node first, left node, then right node last
In Order - left node first, then current node, then right node last
Post Order - left node first, right node second, then current node last

const preOrder = node => 
{
  const traverse = (node, result) => {
    if (!node) result.push(null)
    else {
      result.push(node.data)
      if (node.left) traverse(node.left, result)
      if (node.right) traverse(node.right, result)
    }
  }
  let result = []
  if (!node) return result
  traverse(node, result)
  return result
}

const inOrder = node => 
{
  const traverse = (node, result) => {
    if (node.left) traverse(node.left, result)
    result.push(node.data)
    if (node.right) traverse(node.right, result)
  }
  let result = []
  if (!node) return result
  traverse(node, result)
  return result
}

const postOrder = node => 
{
  const traverse = (node, result) => {
    if (!node) {}
    else {
      if (node.left) traverse(node.left, result)
      if (node.right) traverse(node.right, result)
      result.push(node.data)
    }
  }
  let result = []
  if (!node) return result
  traverse(node, result)
  return result
}

Refer to: https://www.codewars.com/kata/5268956c10342831a8000135

Breadth First Search

Level Recursion - Traverse by each level of a Binary Search Tree

def tree_by_levels(node):
    result = []

    if node is None:
        return result

    level = [node]
    nxt = []

    while len(level) > 0:
        temp = []
        count = 0
        l = len(level)

        while count < l:
            if (len(level) > 0):
                c = level.pop(0)
                if c is None:
                    continue
                temp.append(c.value)
                nxt.append(c.left)
                nxt.append(c.right)
            count += 1

        if len(temp) > 0:
            for x in range(0, len(temp)):
                result.append(temp[x])
        level = nxt

    return result

Refer to: https://www.codewars.com/kata/52bef5e3588c56132c0003bc

Resources and Links

Data Structures: Singly Linked List

A "chain-like", derived, data structure that connects instances head to tail through a next attribute, each instance taking a value.

Attributes

Value
Next - null, nil, none, or another Linked List instance.

Time Complexity

Generally, the average time complexity for most operations will be O(N). One needs to traverse through a Linked List to find the right instances to alter, insert a new instance between, remove, etc.

Average Read: O(N)
Average Add: O(N)

The best case is O(1) - the desired instance can be right at the beginning of the Linked List at the head.

Implementations

// JavaScript
var LinkedList = function(val, next) {
    this.val = val;

    if (next === undefined || next === null) this.next = null;
    else this.next = next;
}

Here, I implement a very simple Linked List. Java's default LinkedList tracks head and tail via index. The implementation below explicitly stipulates a head and tail and uses Node objects (specifically the next property) to create a chain.

// Java
public class Node {
  private Node next;
  private Object data;

  public Node(Object data, Node next) {
    this.data = data;
    this.next = next;
  }

  //... Getters and Setters
}

public class LinkedList {
  private Node head;
  private Node tail;

  public LinkedList() {
    Node tail = new Node(null, null);
    this.head = new Node(null, tail);
    this.tail = tail;
  }

  public LinkedList(Node head, Node tail) {
    this.head = head;
    this.tail = tail;
  }
}

Common Operations

// JavaScript
function append(head, val) {
  if (head == null) head = new LinkedList(val, null);
  else {
    var current = head;
    while (current.next != null) {
      current = current.next;
    }
    current.next = new LinkedList(val, null);
  }
  return head;
};

function prepend(head, val) {
  if (head == null) head = new LinkedList(val, null);
  else {
    var current = head;
    head = new LinkedList(val, current);
  }
  return head;
};

/** Remove by value not by index. */
function del(head, val) {
  let current = head, arr = []

  while (current != null) {
    if (current.val != val) arr.push(current.val)
    current = current.next
  }

  return buildList(arr);
};

/** - Starting At 1 */
function insrt(head, val, position) {
  if (head == null) head = new LinkedList(val, null);
  else {
    var current = head;
    var last = head;
    for (var i = 1; i < position; i++) {
      last = current;
      current = current.next;
    }
    current = new LinkedList(val, current);
    last.next = current;
  }
  return head;
};

function checkCycle(head) {
  if (head == null) return false;
  else {
    var alreadyIn = [];
    var current = head;
    while (current != null) {
      if (alreadyIn.indexOf(current.val) == -1) alreadyIn.push(current.val);
      else return true;
      current = current.next;
    }
    return false;
  }
};

Notes

A LinkedList can be thought of as Tree with only one child element at each node.

Resources and Links

Data Structures: Stack

A "paper-stack" like derived object that's LIFO (last in, first out).

Often implemented with an Array or LinkedList.

Data Structures: Queue

A "waiting line" (British "queue") and derived object that's FIFO (first in, first out).

Often implemented with an Array or LinkedList.

Data Structures: Sets

Most languages provide some implementation (or approximation) of the (the Pure Mathematics) Set Theoretic conception.

In Pure Mathematics, Sets are defined by the following:

As mathematical objects that "contain" other such objects (typically, in the manner specified by the axioms of ZFC Set Theory) - when they are allowed to "contain" themselves (Impredicativity) it leads to the notorious Cantorian Paradoxes (e.g. - Naive Set Theory).
They are denoted by pairs of enclosing curly braces: {, }.
∈ stands for the elemental inclusion operator and A ∈ B is read as Set A is an element of Set B (e.g. - B := { A, ... })
Where the order and duplication of elements are irrelevant and aren't preserved.
Where the identity of two Sets is determined by them having the exactly same elements.
Where the Null Set is always an element of every Set.
Where various common operations like Union, Complementation, Proper Subset, Intersection, and the like are defined using the above definitions.

Programmatically, Sets are generally characterized by the following (slightly weaker) constraints:

The order of their elements isn't preserved.
Their elements are deduplicated.
Set Theoretic Identity and constraints on Impredicativity are respected.

Refer to: Algorithmic Implementations of Common Set Operations

There are some important differences between the above and the pure math conception:

In Set Theory (as a discipline of Mathematics), the Null Set is an element of every Set. This is usually ignored in most languages.
In plain Java, probably the closest native operation to Set Theoretic Intersection in the Collections API is retainAll(). retainAll() however modifies the first Set and Set Theoretic Intersection is a distinct Set. As such retainAll() is not precisely Set Theoretic Intersection.

Note that Apache's Java Common Collection Utils supplies both a customized retainAll() and intersection() methods that return distinct Sets.

Resources and Links

Computer Science: Arguments and Parameters

Often used interchangeably (and often confused).

Parameters are used in the definition of a function or method signature. They are the range of values and types that a function can take.

Arguments are the values that a function takes when being called or invoked.

Original Source of Confusion

From: https://chortle.ccsu.edu/java5/Notes/chap34A/ch34A_3.html - I think the point of confusion arises from the following original terminology:

Formal Parameter — the identifier used in a method to stand for the value that is passed into the method by a caller.
- For example, amount is a Formal Parameter of some generic method processDeposit
Actual Parameter — the actual value that is passed into the method by a caller.
- For example, the 200 used when processDeposit is called is an Actual Parameterr.
- Actual Parameters are often called Arguments

Computer Science: Transient Objects

It's often useful to have "temporary", in-memory objects, that aren't persisted or saved to a database.

Such Transient Objects can be Fields, used in Serializing/Deserializing, validation, etc.

Transient Fields

// Java
@Entity
public class Person {

    @Transient
    private String temporaryNote;

}

Note that in Java, the @Transient annotation and keyword transient accomplish much of the same. Fields and their values are ignored and/or replaced with another value.

If one both implements Serializable and uses the transient keyword in Java, default values are created, persisted, and the original values are stored in a separate file. (This was often used for PII since it separates sensitive data into two parts that have to be reassembled.)

// Java
public class Employee implements Serializable {
    private static final long serialVersionUID = 1L;
    private transient Address address;

    private void writeObject(ObjectOutputStream oos) 
      throws IOException {
        oos.defaultWriteObject();
        oos.writeObject(address.getHouseNumber());
    }

    private void readObject(ObjectInputStream ois) 
      throws ClassNotFoundException, IOException {
        ois.defaultReadObject();
        Integer houseNumber = (Integer) ois.readObject();
        //...
        this.setAddress(a);
    }

    //...
}

# Ruby
class MyModel < ActiveRecord::Base
  attribute :small_int, :integer, limit: 2
end

Data Transfer Objects

Plain Old Java Objects or some other intermediary object (say in Ruby) can be used to then Get and pass data into the relevant domain entity (Hibernate or ActiveRecord, above).

Resources and Links

Computer Science: Pointers

Variables are typically a compound with a name, a declaration keyword, some value setting symbol, and a value.

int myVar = 1;

Pointers point to a Variable’s address / reference in memory.

One can think of a Pointer as a reference to a value in memory or as a memory address.

int *numAddress = & myVar;

Dereferencing the Pointer / address to get the value back.

int derefNum = *numAddress;
myVar == derefNum; // True

Computer Science: Processes and Threads

A Process typically corresponds to a Program.

A Process usually has multiple Threads.

Threads have their own Stack and don't necessarily Synchronize their activities. They can interact in somewhat unpredictable ways, in total isolation, or be Synchronized using:

synchronized
Mutual Exclusion Locks (Mutex)
Semaphors
Atomic Types that are inherently thread-safe and Synchronized.

To illustrate this relationship further:

Consider explicitly creating a Thread or Runnable object in Java or Exec fork in JavaScript: the Process spawns another Threads.
One Process, multiple Threads.

One finer point that’s sometimes forgotten: the Node engine is Single-Threaded and so, one Process is one Thread (under normal circumstances). So, Exec fork, spawn, and Child exec will all create new Single-Threaded Processes.

Computer Science: Heap vs Stack

Objects stored in Heap memory are persisted across all Threads.

Objects stored in Stack memory are available through the execution of a method or function (typically the scope of a function).

It's convenient to think of a Stack as a "local" Heap.

Objects can be stored:

In the Heap
In the Stack without being persisted to the Heap
In both

In C++, Stack memory is automatically managed by the Garbage Collector, and Heap memory requires explicit calls (to say persist something in memory across the lifespan of a specific function).

Computer Science: Hexadecimal Numbers

Fully general algorithm.

// JavaScript
// 0-9
// A-F represent 10-15
const NUMS = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "A", "B", "C", "D", "E", "F"];

const decimalToHexadecimal = (num, remainders = []) => {
  if (num < 0) {
    // Specific to num < 0
    // A trick to convert negative decimals to hex
    num = num + Math.pow(2, 32)
  }

  const r = num % 16
  const d = Math.floor(num / 16)

  remainders.push(NUMS[r])

  if (d < 1) return remainders.reverse().join("")
  return decimalToHexadecimal(d, remainders)
}

const hexToDecimal = hex => {
  let num = 0 
  // Any hexadecimal that begins with 8-F is negative
  if (NUMS.indexOf(hex.charAt(0)) >= 8) num -= Math.pow(2, 32)
  let order = hex.length

  for (let i = 0; i < hex.length; i++) {
    order--;
    var n = NUMS.indexOf(hex.charAt(i));
    num += n * Math.pow(16, order);
  }

  return num;
}

Int to Hex

Int to Hex.

Hex to Binary in JS:

// JavaScript
var hexToBinaryMap = {
    "0": "0000",
    "1": "0001",
    "2": "0010",
    "3": "0011",
    "4": "0100",
    "5": "0101",
    "6": "0110",
    "7": "0111",
    "8": "1000",
    "9": "1001",
    "a": "1010",
    "b": "1011",
    "c": "1100",
    "d": "1101",
    "e": "1110",
    "f": "1111"
}

function intToHex(num) {
    var hex = parseInt(num, 10).toString(16);
    if (hex == "0") hex = "0000";
    return hex;
}

function hexTo16BitBinary(hexString) {
    var l = hexString.length, innerString = "", result = "";
    for (var i = 0; i < l; i++) {
        innerString += hexToBinaryMap[hexString.charAt(i)];
    }
    return innerString;
}

Computer Science: Signed and Unsigned Numbers

Signed numbers can be positive, negative, or zero (The Naturals, full Reals).

Unsigned can’t (Positive Ordinals, etc.).

Important Terminology

Radix – number of unique digits used to represent a number
- 16 bit binary – base 2 (radix 2) since it uses 2 numbers in the base.
- Hexadecimal is radix 16 since it uses base 16
Base – the number system (2, 10, decimal, etc.).
- Refer to Log notes above.
- Implicitly base 10 or 2 otherwise.
Mantissa – Two meanings:
- The Significand (IEEE 754 double-precision 52-bit or 53-bit with hidden bit Significand) and the numbers following a decimal (x.yyyyyyy).
- The first sense defines the number of bits that are used to represent the significant numbers (including a sign or not) that are multiple against some power value (e.g. – [the binary representation of] 12345 x 10-5).
- The second sense expresses the intent of the first operation.

Computer Science: Bitwise Operators

Convert to Binary - convert some Number to the Binary representation of that same Number:

 // Java
 int x = 1000;
 Integer.toBinaryString(x);

 // JavaScript
 const x = 1000;
 const xx = x.toString(2);

~ - Bitwise Not (negation, complementation) - inverts the value of every Binary String index: 0 if 1 and vice-versa.
- 4 is 0100 (Base 10 to Binary)
- ~0100 = 1011
- ```
 System.out.println(Integer.toBinaryString(~4)); // 1011
```
& - Bitwise And - compares every Binary String index and returns 1 if both characters are 1 otherwise 0.
- 5 is 0101 (Base 10 to Binary)
- 0100 & 0101 = 0100
- ```
 System.out.println(Integer.toBinaryString(4 & 5)); // 0100
```
| - Bitwise Or - compares every Binary String index and returns 1 if at least one character is 1 otherwise 0.
- 0100 | 0101 = 0101
- ```
System.out.println(Integer.toBinaryString(4 | 5)); // 0101
```
^ - Bitwise Xor ("Exclusive Or")- compares every Binary String index and return 1 if exactly one character is 1 otherwise 0.
- 0100 ^ 0101 = 0001
- ```
System.out.println(Integer.toBinaryString(4 ^ 5)); // 0001
```
<< - Bitwise Left Shift (unsigned) - increases the length of the Binary String by the shift value, right-fills with 0.
- 1 << 3 = 1000
- ```
System.out.println(Integer.toBinaryString(1 << 3)); // 1000
```
>> - Bitwise Right Shift (unsigned) - shifts a Binary String to the right by the shift value, fills with 0 if there's empty space
- 0001 >> 3 = 0000
- Alternatively: 0001 >> 3 = 0
- ```
System.out.println(Integer.toBinaryString(0001 >> 3)); // 0000
```

Note that Java will truncate the leading leftmost 0.

Resources and Links

C++: General Concepts

A compiled, Object Oriented, statically typed programming language that predates most others. Some quirks:

C++ programs will execute even if they don't compile!
C++ supports Pointers and Dereferencing.
C++ divides files into Class Definitions (suffixed .h files) and their implementations (suffixed .cpp files).
C++ often requires more explicit use of Garbage Collecting and memory management (especially w.r.t. the Heap).

Runtime Environment

Use gcc through Xcode (if on a Mac) and cmake for compilation and executation of C++ code:

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.0 (clang-1100.0.33.16)
Target: x86_64-apple-darwin19.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ cmake --version
cmake version 3.16.6

CMake suite maintained and supported by Kitware (kitware.com/cmake).

Console Out

cout << “hello world”;

Note that the << operator is the concatenation operator (. or + in other languages) it's outputs data to the console. >> receives data (from say cin).

Resources and Links

Code samples:

https://github.com/Thoughtscript/cplusplus-coursera/tree/master/examples

C++: Files and OOD

There are no Interfaces or Abstract Classes in C++. However:

filename.h a Class Definition that’s pre-compiled (since it's not likely to change).
filename.cpp the implementation of that Class Definition (with full method definitions/implementations).
:: is the scope resolution operator and defines the implemented methods of the class definition.
: is the subclassing operator (equivalent of the extends keyword in Java).

Constructors

C++ Constructors come in a few flavors:

Default Constructors - C++ supported default constructors that are automatically supplied for each Class. It's convenient to think of these as akin to the @NoArgs, @AllArgs annotations in Spring.
Customized Constructors - developer-supplied and defined constructors.
Copy Constructors - used internally or explicitly invoked to initialize a new Object with the values of another member of the same Class.

Refer to: https://en.cppreference.com/w/cpp/language/default_constructor, https://en.cppreference.com/w/cpp/language/copy_constructor

For example, given ExampleClass.h, ExampleClass.cpp, and main.cpp:

ExampleClass.h:

// header file - think interface/abstract class skeleton - not likely to change and doesn't need to be recompiled
#ifndef Example_Class_H // pragma guard - used at compile time to prevent redundant appending/prepending of compiled code
#define Example_Class_H // if not already defined, add - otherwise it will be ignored

namespace ExampleNamespace {
    class ExampleClass {
        public:
            // even explicitly specifying constructor here defines a custom
            ExampleClass();
            ExampleClass(int _a, int _b);
            int a, b;
            void exampleMethod();
    };
}

#endif

ExampleClass.cpp:

// example_class implementation
#include <iostream> // header in standard library
#include "ExampleClass.h" // header in custom library

using namespace ExampleNamespace; // now you don't have to prepend the namespace when calling methods, etc.
using namespace std;

// implement class method without class syntax here
void ExampleNamespace::ExampleClass::exampleMethod() {
    a = 5;
    b = 100;
    std::cout << "I'm a console out message from ExampleClass exampleMethod()" << endl;
}

// custom default constructor
ExampleNamespace::ExampleClass::ExampleClass() {
    a = 100;
    b = 100;
}

// custom constructor
ExampleNamespace::ExampleClass::ExampleClass(int _a, int _b) {
    a = _a;
    b = _b;
}

main.cpp:

#include <iostream>
#include "ExampleClass.h"

using namespace ExampleNamespace;
using namespace std;

int main() {
    try {
        ExampleClass ec; // custom default constructor
        std::cout << &ec << " with " << ec.a << " " << ec.b << std::endl;

        ExampleClass ecc(1,1); // custom constructor
        std::cout << &ecc << " with " << ecc.a << " " << ecc.b << std::endl;

        ExampleClass * eccc = new ExampleClass; // pointer with custom default constructor
        std::cout << eccc << " with " << (*eccc).a << " " << (*eccc).b << std::endl;
        delete eccc;

        ExampleClass * ecccc = new ExampleClass(111,111); // pointer with custom constructor
        std::cout << ecccc << " with " << (*ecccc).a << " " << (*ecccc).b << std::endl;
        delete ecccc;

        // copies - keeps different addresses
        ExampleClass copyExample;
        copyExample.a = 1000;
        copyExample.b = 1000;

        ExampleClass otherCopyExample;
        otherCopyExample.a = 1111;
        otherCopyExample.b = 1111;

        std::cout << copyExample.a << " " << copyExample.b  << " at: " << &copyExample << " " << otherCopyExample.a << " " << otherCopyExample.b << " at: " << &otherCopyExample << std::endl;
        copyExample = otherCopyExample;
        std::cout << copyExample.a << " " << copyExample.b  << " at: " << &copyExample << " " << otherCopyExample.a << " " << otherCopyExample.b << " at: " << &otherCopyExample << std::endl;

        // memory assignment - note how the memory addresses for the variables remain distinct despite assigning their pointers to each other.
        ExampleClass x;
        x.a = 111;
        x.b = 111;
        ExampleClass * xx = &x;

        ExampleClass y;
        y.a = 1000;
        y.b = 1000;
        ExampleClass * yy = &y;

        std::cout << x.a << " " << x.b  << " at: " << &x << " " << xx << " " << y.a << " " << y.b << " at: " << &y << " " << yy << std::endl;
        xx = yy;
        std::cout << x.a << " " << x.b  << " at: " << &x << " " << xx << " " << y.a << " " << y.b << " at: " << &y << " " << yy << std::endl;
        std::cout << (*xx).a << " " << (*xx).b << " at: " << xx << " " << (*yy).a << " " << (*yy).b << " at: " << yy << std::endl;

        // references
        ExampleClass refCopyExample;
        refCopyExample.a = 1000;
        refCopyExample.b = 1000;

        ExampleClass & otherRefCopyExample = refCopyExample;

        std::cout << refCopyExample.a << " " << refCopyExample.b  << " at: " << &refCopyExample << " " << otherRefCopyExample.a << " " << otherRefCopyExample.b << " at: " << &otherRefCopyExample << std::endl;


    } catch (const std::exception &e) {
        std::cout << e.what() << std::endl;
    }

    return 0;
}

Class Object Assignments

Remember that objects created in the Stack do not automatically persist in the Heap. One illuminating topic is how objects in two distinct functions can be assigned.

ExampleClass methodOne() {  
    ExampleClass ec;
    return ec;  
}

int main() {  
    ExampleClass ecc;
    ecc = methodOne();  
}

Will the above throw an error? No, the following process is performed:

Default Constructor called in both methodOne() (for ec) and main() (for ecc).
Copy Constructor is called when returning methodOne() (this link two discrete events on the Stack).
Assignment Operator is called to copy the "innards" from ec to ecc (values are copied from ec to ecc).

Access modifiers

Can be declared in classes (for encapsulated class fields) using the following:

// header file - think interface/abstract class skeleton - not likely to change and doesn't need to be recompiled
#ifndef Example_Class_One_H // pragma guard - used at compile time to prevent redundant appending/prepending of compiled code
#define Example_Class_One_H // if not already defined, add - otherwise it will be ignored

namespace ExampleNamespace {
    class ExampleClassOne {
        public:
            int num;
            void exampleMethodOne();
    };
}

#endif

Class Methods

Class methods don't have to be explicitly declared within the body of a Class Definition. (It's convenient to think of this as akin to a Function declaration that's place above any class keyword in ES6+.)
Separate Class implementation files can therefore be omitted.
Similarly, the Class Definition file needn't contain a class keyword at all.

Example One

ExampleClassOne.h:

// header file - think interface/abstract class skeleton - not likely to change and doesn't need to be recompiled
#ifndef Example_Class_One_H // pragma guard - used at compile time to prevent redundant appending/prepending of compiled code
#define Example_Class_One_H // if not already defined, add - otherwise it will be ignored

namespace ExampleNamespace {
    class ExampleClassOne {
        public:
            int num;
            void exampleMethodOne();
    };
}

#endif

main.cpp:

// example_class implementation
#include <iostream> // header in standard library
#include "ExampleClassOne.h" // header in custom library

using namespace ExampleNamespace; // now you don't have to prepend the namespace when calling methods, etc.
using namespace std;

// implement class method without class syntax here
void ExampleClassOne::exampleMethodOne() {
    cout << "I'm a console out message from ExampleClassOne exampleMethodOne()" << endl;
}

// executable code (must be wrapped in main method)
int main() {
    try {
        ExampleClassOne exampleOne;
        cout << "Review the random number assigned here: " << exampleOne.num << endl;
        exampleOne.num = 2;
        exampleOne.exampleMethodOne();
        cout << exampleOne.num << endl;

    } catch (const std::exception &e) {
        std::cout << e.what() << std::endl;
    }

    return 0;
}

Refer to: https://github.com/Thoughtscript/cplusplus-coursera/tree/master/examples/6%20-%20class

Example Two

add.h:

#ifndef Example_Class_H
#define Example_Class_H

int add(int x, int y)
{
    return x + y;
}

#endif

main.cpp:

#include <iostream>
#include "add.h"

int main() {
    try {
        std::cout << "The sum of 3 and 4 is " << add(3, 4) << '\n';
    } catch (const std::exception &e) {
        std::cout << e.what() << std::endl;
    }
    return 0;
}

Refer to: https://github.com/Thoughtscript/cplusplus-coursera/tree/master/examples/3%20-%20dependency

Inheritance

Given a pair of Class Definitions and their implementations: BaseClass and SuperClass.

SuperClass.h:

#include <iostream>
#include "SuperClass.h"

using namespace ExampleNamespace;
using namespace std;

void SuperClass::superClassMethod() {
    num = 500;
    std::cout << "superClassMethod() called in SuperClass " << num << std::endl;
}

SuperClass.cpp:

#ifndef SUPER_CLASS_H
#define SUPER_CLASS_H

namespace ExampleNamespace {
    class SuperClass {
    public:
        int num;
        virtual void superClassMethod();
    };
}

#endif

BaseClass.h:

#ifndef BASE_CLASS_H
#define BASE_CLASS_H

#include "SuperClass.h"

// Specify an associated Namespace - this is akin to a package in Java
namespace ExampleNamespace {
    class BaseClass: virtual public SuperClass {
        public:
            int num;
            void baseClassMethod();
            // Note that this may throw a warning - it can be ignored
            // warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
            void superClassMethod() override;
            void superEquivalentMethod();
    };
}

#endif

BaseClass.cpp:

#include <iostream>
#include "BaseClass.h"

using namespace ExampleNamespace;
using namespace std;

void BaseClass::baseClassMethod() {
    num = 3;
    std::cout << "baseClassMethod() " << num << std::endl;
}

void BaseClass::superClassMethod() {
    num = 7;
    std::cout << "superClassMethod() override " << num << std::endl;
}

void BaseClass::superEquivalentMethod() {
    SuperClass::superClassMethod();
    std::cout << "superEquivalentMethod() called in BaseClass " << num << std::endl;
}

main.cpp:

// simple classless executable with function.

#include <iostream>
#include "BaseClass.h"

using namespace ExampleNamespace;

int main() {
    try {
        ExampleNamespace::BaseClass bc;
        bc.baseClassMethod();
        bc.superClassMethod();
        bc.superEquivalentMethod();
    } catch (const std::exception &e) {
        std::cout << e.what() << std::endl;
    }
    return 0;
}

Refer to: https://github.com/Thoughtscript/cplusplus-coursera/tree/master/examples/8%20-%20inheritance

Resources and Links

Code samples:

C++: Memory

Generally, C++ requires and offers more precise control over memory use in terms of Garbage Collection, variable declaration, and memory addressing.

Garbage Collection

No automatic Garbage Collection (like in Java) - variables are not automatically removed from memory.
Unlike Java, C++ distinguishes between Heap and Stack memory (for Garbage Collection). In Java, application memory is allocated and automatically Garbage Collection in the Heap (which exists as an environment resource). Java's Stack is a derivative object implemented as an LIFO Dequeue and created within the application.
C++ will automatically clear Stack memory and variables (e.g. - variables within the scope of a completed function will be removed from the Stack).
For Heap memory (memory that persists longer than the lifespan of a specific function), using delete, free(), malloc(), etc. is required.
Use new to allocate memory to Heap and delete to remove (recommended). Use new if a variable must be passed between functions.

Pointers and References

Consider:

int num = 0;

Variables hold values.
Reference operator gets the address from a variable holding a value.

&num //address of value of num

Pointer variables hold addresses.
Note: The & symbol can also be used in variable declaration to essentially create another name or alias for a variable already declared.

int* pointer_num = &num; //address of value of num

Dereferencing returns the value of the address stored in a pointer variable.

*pointer_num; //0

Tip: think of ** as being the same as no * (the two cancel out) - akin to double-negation elimination.

Example

#include <iostream>

int main()
{
    try {
        // ------------------- variable with value -------------------
        int num = 100;
        // note using std::cout explicitly here instead of the using keyword at top of file
        std::cout << "num " << num << std::endl;

        // ------------------- pointer variable with a reference to the address of the variable above  -------------------
        int *numAddress = &num;
        std::cout << "numAddress " << numAddress << std::endl;

        // ------------------- dereference the address to get the value back -------------------
        int derefNum = *numAddress;
        std::cout << "derefNum " << derefNum << std::endl;
        *numAddress = 42;
        std::cout << "numAddress " << numAddress << std::endl;
        std::cout << "*numAddress " << *numAddress << std::endl;

        // ------------------- Reference variables -------------------
        int & refVar = derefNum;
        std::cout << "refVar to derefNum " << refVar << std::endl;

        // ------------------- heap example #1 -------------------
        // declare a pointer variable using new keyword - which automatically (always) assigns memory to the heap
        int * exampleA = new int;
        std::cout << "Initialized to last value on heap: " << exampleA << " " << *exampleA << std::endl;
        delete exampleA;

        // -------------------  heap example #2  -------------------
        // declare a pointer variable and allocate a memory address in heap
        int * heapVariable = (int*) malloc(1);
        // assign a value to the pointer variable that doesn't exceed the specified size
        heapVariable[0] = 45;
        std::cout << "Heap assigned value " << heapVariable[0] << std::endl;
        std::cout << "Heap pointer variable / address " << heapVariable << std::endl;
        // return the allocated memory block back to the heap
        free(heapVariable);

        // ------------------- heap example #3 -------------------
        // declare a pointer variable using new keyword - which automatically assigns memory to the heap
        int * newVar = new int;
        std::cout << "Note the value initialized to is " << newVar << " " << *newVar << std::endl;
        *newVar = 1000;
        std::cout << "newVar " << *newVar << " at " << newVar << std::endl;
        // declare a pointer variable assigning NULL
        int * nullVar = NULL;
        // use delete keyword only for variables declared with new keyword or NULL
        delete newVar;
        delete nullVar;

        // ------------------- null pointer versus NULL -------------------
        // NULL is a value that can be assigned to a variable
        // null pointer keyword specifies a null address - technically, 0x0
        int * pointerVar = nullptr;
        // Cannot access nor delete - will throw error or exit code 11 if you attempt either

    } catch (const std::exception& e) {
        std::cout << e.what() << std::endl;
    }

    // main() must always return an exit code
    return 0;

}

C++: Template Functions

C++ supports Template Functions which are akin to using Java Generics in Method definitions.

Example

ExampleClass.h:

#include <iostream>

#ifndef EXAMPLE_CLASS_H
#define EXAMPLE_CLASS_H

using namespace std;

namespace ExampleNamespace {

    // only one type need be flexibly declared
    template<typename T> class ExampleClass {
    public:
        int num;
        T flexibleVar;

        // best to define these in the same class with template<typename T> declaration
        void exampleMethodOne() {
            cout << "exampleMethodOne() " << flexibleVar << " " << typeid(flexibleVar).name() << endl;
        }

        T flexibleMethodOne(T a) {
            cout << "flexibleMethodOne() " << a << " " << typeid(a).name() << endl;
            return a;
        }

        T flexibleMethodTwo(T a, T b) {
            T result = a + b;
            cout << "flexibleMethodTwo() " << result << " " << typeid(result).name() << endl;
            return a + b;
        }
    };
}

#endif

main.cpp:

#include <iostream>
#include "ExampleClass.h"

using namespace std;
using namespace ExampleNamespace;

// Within the definition of a Function
template<typename V>
V standaloneExampleMethod(V x) {
    return x;
}

int main() {
    try {

        ExampleClass<string> ec;
        string random = "I am a random string";
        ec.flexibleVar = random;
        ec.num = 0;

        ec.exampleMethodOne();
        ec.flexibleMethodOne("text");
        ec.flexibleMethodTwo("hello","world");
        std::cout << "My values are " << ec.flexibleVar << " " << ec.num  << '\n';

        ExampleClass<int> ecc;
        ecc.flexibleVar = 5;
        ecc.num = 100;
        ecc.flexibleMethodTwo(1,2);
        std::cout << "My values are " << ecc.flexibleVar << " " << ecc.num  << '\n';

        string a = standaloneExampleMethod("hello");
        int b = standaloneExampleMethod(5);
        std::cout << "Flexible template values " << a << " " << b  << '\n';

    } catch (const std::exception &e) {
        std::cout << e.what() << std::endl;
    }

    return 0;
}

Ruby: General Concepts

nil is the nullish value keyword.
Ruby is an inherently Synchronous language. There's nothing that corresponds to a native Promise.
Modules are Mixins that can be included in a Class. This also provides Java-like Interface and Abstract Class reuse.
All Function, Method, and Lambda-types are Closures.

Ruby: Hashes and Arrays

Hash

A Hash is a dict / array (since they are integer-indexed collections) / map equivalent.

Named Key - Value (e.g. - Objects in JS, dicts in Python).

# hash_example = {}
# hash_example = Hash.new
hash_example = Hash[]
hash_example['a'] = n
hash_example[:b] = 100

Array

Array - an Array proper (e.g. - Key by index Value).

Expands in size - ArrayList like (Java).

arr_example = Array.new() 
arr_example = []
arr_example.push(1)
arr_example.push(2)
arr_example.push(3)
p arr_example.first
p arr_example.last

In Ruby, Array length and size() accomplish the same.

size() (the method) is an alias for the field length.

Added for uniformity since many languages use length (Java, JavaScript) while C++ uses size().

Refer to the Documentation.

Resources and Links

Code samples:

Ruby: Interceptors

before_action - provides the same functionality as a Java Interceptors, executes a function prior to other actions being performed within an HTTP handler.

Ruby: Object Oriented Design

Inheritance

class Animal
  attr_accessor :name

  def initialize(name)
    @name = name
  end

  def speak
    "Hello!"
  end
end

class GoodDog < Animal
  def initialize(name, color)
  super(name)
    @color = color
  end

  def speak
    super + " from GoodDog class"
  end
end

Resources and Links

Code samples:

https://github.com/Thoughtscript/ruby_review/tree/master/examples/6-ood

Ruby: Exception Handling

Use fail instead of raise in circumstances where an Exception will not be handled and rethrown as something more specific.
Use raise otherwise.
rescue is the relevant "catch" keyword.

Examples

begin
  p hash_example['a']['b']
rescue Exception
  p "hash_example['a']['b'] throws an exception" 
end

begin
  p hash_example['a']['b']
rescue Exception
  p "hash_example['a']['b'] throws an exception" 
ensure
  p "I'm a finally block"
end

Ruby: Reminders

Not "gotcha's" generally speaking. Things I'm prone to forget! Refresh these until they are cemented in memory.

each_with_index:

a.each_with_index { |item, index|
  p %Q(#{item} at index: #{index})
}

Lambda proc with no variable name (anonymous function called with an actual parameter or argument):
```
-> (arg) {p arg +1 }.call(2)
```

https://docs.google.com/presentation/d/1cqdp89_kolr4q1YAQaB-6i5GXip8MHyve8MvQ_1r6_s/edit#slide=id.g2048949e_0_38

https://www.toptal.com/ruby/interview-questions

GitHub repo: https://github.com/Thoughtscript/ruby_review

Consider the example of checking the index of an M x N Array. Specifically, where some row r may be Out of Bounds.

Or, alternatively, checking if a nested field exists on an Object.

Three ways of checking that include are given as follows.

The Overly Verbose Way

if account && account.owner && account.owner.address
    # ...
    # => false
    # => nil
end

ActiveRecord Try

if account.try(:owner).try(:address)
    # ...
    # => false
    # => nil
end

if  account&.owner&.address
    # ...
    # => nil
    # => undefined method `address' for false:FalseClass`
end

With Hashes

Given:

hash_example = Hash[]
hash_example['a'] = n
hash_example[:b] = 100

To check for nested a > :b:

hash_example['a']&.try(:b)
hash_example['a']&.try(:[], :b)

Array#dig and Hash#dig

Consider:

address = params[:account].try(:[], :owner).try(:[], :address)
# or
address = params[:account].fetch(:owner) { {} }.fetch(:address)

address = params.dig(:account, :owner, :address)

Resources and Links

https://mitrev.net/ruby/2015/11/13/the-operator-in-ruby/

Code samples:

https://github.com/Thoughtscript/ruby_review/tree/master/examples/16-safe_navigation

Ruby: Truthy Evaluations

In Ruby, only false and nil are evaluated to false.

0, [], etc. all evaluate to true.

Ruby: Visibility and Access

@@ - Class Variable, defines a field that's synchronized between all instances of the Class. For example, counting the number of instantiated copies of a Class that have been created since Application start.
@ - fields must be set and got using getters and setters, same as self.
attr_accessor - fields can be accessed directly with dot notation.
private - a keyword that can be used to indicate that every Closure definition in the scope below be given the private access visibility modifier.

Example

class ExampleClass
  attr_accessor :field_one

  def set_and_get_field_two(arg)
    ## Does not have to be declared as attr_accessor
    ## But cannot be directly accessed in public
    @field_two = arg
    p @field_two
  end

  def get_field_two()
    p @field_two
  end

  def get_field_one
    ## These are the same
    p @field_one
    p self.field_one

    example_private(arg1: @field_one)
  end

  # Everything below is given the 'private' access modifier
  private

  def example_private(arg1:)
    p arg1
  end
end

## Call 
### Constructor
e = ExampleClass.new
### Set attr_accessor field
e.field_one = 2
### Getter for @
e.get_field_one

Resources and Links

Code samples:

https://github.com/Thoughtscript/ruby_review/blob/master/examples/12-access/main.rb

Ruby: Common Rails Commands

Start

rails db:create
rails db:migrate
rake db:seed
rails server --binding=127.0.0.1

By default, the Ruby on Rails serve will serve from: http://localhost:3000/

Migrations

bin/rails generate migration ExampleMigration
rails db:migrate
# rake db:migrate

Reset DB

# run migration and seeding
rails db:setup 
# rails db:create
# rails db:migrate
# rake db:seed

rails db:reset

Create Model and Table

rails g model Dinosaur name:text
rails g model BabyDino name:text

Create Controller

rails g controller Dinosaurs

Resources and Links

Code samples:

https://github.com/Thoughtscript/rrr_2022

Ruby: Active Record

Active Record FK Example

Schema

ActiveRecord::Schema[7.0].define(version: 2022_09_05_182026) do
  # These are extensions that must be enabled in order to support this database
  enable_extension "plpgsql"

  create_table "baby_dinos", force: :cascade do |t|
    t.bigint "dinosaur_id"
    t.text "name"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
    t.index ["dinosaur_id"], name: "index_baby_dinos_on_dinosaur_id"
  end

  create_table "dinosaurs", force: :cascade do |t|
    t.text "name"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
  end

end

From: https://github.com/Thoughtscript/rrr_2022/blob/main/_ruby/web/db/schema.rb

Models

class BabyDino < ApplicationRecord
    attribute :name, :string

    # Does not need to be explicitly set
    # attribute :id, :integer
    # self.primary_key = :id

    # Does not need to be explicitly set
    # attribute :dinosaur_id, :integer
    belongs_to :dinosaur, class_name: "Dinosaur", inverse_of: :baby_dinos

    validates :name, presence: true
end

From: https://github.com/Thoughtscript/rrr_2022/blob/main/_ruby/web/app/models/baby_dino.rb

class Dinosaur < ApplicationRecord  
    # Remember that Rails ActiveRecord uses attributes here!
    # Distinct from DTO's.  
    attribute :name, :string

    # Does not need to be explicitly set
    # #attribute :id, :integer
    # self.primary_key = :id

    has_many :baby_dinos, inverse_of: :dinosaur

    validates :name, presence: true

    def speak
      "Hello!"
    end
end

From: https://github.com/Thoughtscript/rrr_2022/blob/main/_ruby/web/app/models/dinosaur.rb

Java: Comparisons

Primitive Comparison (==) - used to compare the equivalence of two Primitive Data Types.
Object Comparison (equals()) - used to compare the equivalence of two Objects.

String a = "abc";
String b = "abc";

(a == b); //true - both point to the same reference
a.equals(b); //true - both are the same object value sand reference

Type Checking

Applies to descendants of the Object Class:

Person p = new Person();
System.out.println(p instanceof Person) // true

Comparators

Recall that comparisons typically return:

-1 - some A shoud precede some B
0 - there's no reason for A not to precede some B
1 - some B should precede some A

In the example below, a Comparator is implemented as a Lambda to sort Products per the following:

Sort by Availability (boolean) descending
If tied, sort by DiscountedPrice (Double) ascending
If still tied, sort by Id (Long) ascending

List<Product> products = new ArrayList<Product>();

products.sort((a, b) -> {
    boolean A = a.getAvailability();
    boolean B = b.getAvailability();
    if (A && !B) return -1;
    if (B && !A) return 1;

    Double AA = a.getDiscountedPrice();
    Double BB = b.getDiscountedPrice();
    if (AA < BB) return -1;
    else if (BB < AA) return 1;

    Long AAA = a.getId();
    Long BBB = b.getId();
    if (AAA < BBB) return -1;
    else if (BBB < AAA) return 1;
    return 0;
});

Java: General Concepts

Java Virtual Machine - run Java on any machine or underlying environment.
- Abstracts the Java runtime environment so Java can be executed uniformly anywhere.
Java Heap, Stack, and Garbage Collecting
- Heap - shared memory allocated for use by the JVM and applications running within it.
- Garbage Collecting - the automatic or manually configured/triggered periodic removal of items in memory.
Write Time, Compile Time, and Runtime
- Checked Exceptions and Static methods are handled/checked at Write Time, Compile Time
- Unchecked Exceptions and Non-Static methods are handled/created/checked at Runtime
.java files, compiled Byte Code, and .class files
Impure Object Oriented Design - Primitive Data Types don't inherit from the top-level Object Class.
Reference Types - wrappers for the Primitive Data Types that inherit from the Object Class and therefore can take null values, .stringify(), etc.
Statically Typed - Types are strictly enforced in Parameters, Arguments, Method Signatures, return Types. Limited Auto-boxing.

Top Java 8 Features

Lambdas and Functional Interfaces
CompletableFuture
Stream API

Top Features Since Java 8

sealed keyword
Records
Virtual Threads
Better Heap and JVM memory management.

Java: References

Shallow Copying in Java: set a value without using the new keyword for anything that's not a Primitive Data Type.
Java has no concept like Pass by Reference (unlike JavaScript) but there are some quirks.

Pass by Reference(-ish)

Given:

class ReferenceTest {
    int num;

    ReferenceTest(int x) {
        num = x;
    }

    ReferenceTest() {
        num = 0;
    }
}

public class ReferenceExamples {

    // Remember that Java scope can run counter-intuitively against the expected value.
    public static void examples() {
        ReferenceTest rt = new ReferenceTest(20);
        updateNewReference(rt);
        System.out.println("I'm the outer scope: " + rt.num); // 20 - returns the supplied value above, not 50
        update(rt);
    }

    public static void updateNewReference(ReferenceTest rt) {
        // Restricted by scope, the new value is set within update()
        rt = new ReferenceTest();
        rt.num = 50;
        System.out.println("I'm the inner scope - new reference: " + rt.num);
    }

    public static void update(ReferenceTest rt) {
        rt.num = 50;
        System.out.println("I'm the inner scope - same reference: " + rt.num);
    }
}

Calling examples() will produce the following:

I'm the inner scope - new reference: 50
I'm the outer scope: 20
I'm the inner scope - same reference: 50

Remember to:

Don't create a new reference in memory (via the new keyword) if you just want to modify the value of something you've passed.
Return the exact item you want to return if you intend on reusing a variable.

Resources and Links

https://www.interviewbit.com/java-interview-questions/#does-java-pass-by-value-or-pass-by-reference

Java: Beans

Beans are Encapsulated, reuseable, resources that are created and typically initialized as Singletons in an Application Context.

Some specialized Beans (like the Java EE Message Driven Bean) are not Singletons and respond to triggering messages instead:

They don't maintain a Synchronized or common state throughout the Application.
Today, Message Driven Beans have been widely replaced by other Publish-Subscribe, Event Listening, or Streaming alternatives.

They can then be used anywhere within the Application provided they are correctly configured to do so.

Lifecycle

Within Spring, (customized) Beans are configured using the @Bean annotation within a @Configuration Configuration Class. In Java EE, Beans were traditionally defined using an XML file (and are often defined today using annotations similar to the way that Servlets once were nearly universally defined using XML files and are now often defined programmatically using annotations).

They are then Injected into a Service (@Service) or Component (@Component) typically using the @Autowired annotation (Decorator). Spring Components are Beans themselves (and Services are a kind of Component). Spring knows to look for Components automatically from the @SpringBootApplication or @EnableWebMvc annotation.

Beans are then Initialized at Run Time.
Services that use them as a dependency can modify their state, make calls using their methods, etc.
They are destroyed when the Application Context is shut down or destroyed.

Java: Object Oriented Design

Java uses a top-level Object Class that all non-primitive Types inherit from.

Object Oriented Concepts

Java uses:

Classes (templates, types, or kinds of things that Objects are or that Objects instantiate)
Objects (specific instances, copies, or particular instantiations of Classes)

public class Example {
    private String id;

    // non-parameterized constructor
    public Example() {}

    // parameterized constructor
    public Example(String id) {
        setId(id);
    }

    // getters and setters
    public String getId() {
        return this.id;
    }

    public void setId(String id) {
        this.id = id;
    }
}

Example x = new Example();
Example x = new Example("fsfsf");

Encapsulation

Encapsulation refers to enclosing all the functionalities of an object within that object so that the object’s internal workings (its methods and properties) are hidden from the rest of the application.

Getters and Setters, Access Modifiers, and Packages are some of the primary ways Encapsulation is achieved in Java.

Encapsulation: boundaries, visibility, and access are restricted.

Aggregation

Aggregation refers to one Class having another Class as a field or as belonging to another Class as nested Classes.

Classes can be defined and combined within Classes, a kind of nesting.

Instantiating the Inner Class results in the Outer Class also being instantiated.

public class A {
    //...
}

public class B {
    A a;
}

A nested Class may need to be Static in order to effectively outlive the wrapping Outer Class. Failing to do so can introduce memory leaks.

public interface WidgetParser {
    Widget parse(String str);
}

public class WidgetParserFactory {
    public WidgetParserFactory(ParseConfig config) {
        //...
    }

    public WidgetParser create() {
        new WidgetParserImpl(//...);
    }

    private class WidgetParserImpl implements WidgetParser {
        //...

        @Override 
        public Widget parse(String str) {
            //...
        }
    }
}

Since WidgetParserImpl isn't Static, if WidgetParserFactory is discarded after WidgetParser is created, memory leaks can ensue. Using the static keyword here implies that no instances of WidgetParserImpl need to be instantiated at all (and hence, can avoid the issue of lingering nested Objects).

From: https://www.toptal.com/java/interview-questions and https://www.javatpoint.com/static-nested-class

Note that Composition is a specific, stronger, variant of Aggregation where both the Outer and Inner Classes are tightly coupled: they are created and destroyed together.

From: https://www.geeksforgeeks.org/association-composition-aggregation-java/

Inner and Outer Class Initialization

public class A {
    public class B {
        public String runMe() {
            //...
        }
    }
}

A outerObject = new A();
A.B b = outerObject.new B();
System.out.println(b.runMe());

From: https://docs.oracle.com/javase/tutorial/java/javaOO/nested.html

Refer to: https://github.com/Thoughtscript/java-refresh-notes/blob/master/src/io/thoughtscript/refresh/Main.java

Inheritance

Inheritance:

Classes can be subclassed (e.g. - kinds, kinds of a kind).
The attributes available on parent Classes or prototypes are available on their children.
Inheritance occurs via the extends keyword. Java does not support multiple inheritance in Classes (but it does for Interfaces).

Remember that basic initialization obeys the following general schema:

[Superclass] x = new [Type Inheritor of Superclass]();

super:

Use the super keyword to access Superclass properties.
Call super() to invoke the Superclass constructor (which can be parametized).
Or, use super to invoke a parent Class method like so Collection.super.stream().skip(1);

Polymorphism

Polymorphism: an Object may belong to multiple Classes or exhibit multiple kinds.

Since Java does not directly support Multiple Inheritance in Classes, Polymorphism with respect to Class Inheritance is only accomplished by extending a Class and implementing one or more Interfaces, implementing two or more Interfaces, or by extending a Class that is also extending many other Classes.

Thus, a Class that implements multiple Interfaces is said to exhibit Polymorphism.

public interface Mammal extends Animal, Biologic {
    //...
}

public class Adam extends Human implements Mammal, Developer {
    //...
}

Records

Records - specified by the record keyword - syntactic sugar to specify an immutable POJO / DTO.

Example:

record Car(int seats, String color) {}

Automatically generates Getters and initializes field values.
Automatically generates the AllArgs Constructor.
Can define instance methods and custom Constructors.
Records cannot use an explicit extends keyword:
- All Record Classes are final, so we can’t extend it.
- All Record Classes implicitly extend java.lang.Record Class.
All the fields specified in the record declaration are final.

Resources and Links

Code samples:

Java: Enums

Basic Example

Given:

public class Pet {

    public enum PetType {DRAGON, DOG, CAT, BULL, JARBUL}

    private PetType type;


    public long getPet_id() {
        return pet_id;
    }

    public void setPet_id(long pet_id) {
        this.pet_id = pet_id;
    }

    //...
}

Pet pet = new Pet();
pet.setType(Pet.PetType.DRAGON);

Advanced Topics

Remember that you can define:

Fields
Methods
Constructors

within Enums.

As such one can also implement Singletons.

Resources and Links

https://www.baeldung.com/a-guide-to-java-enums

Code samples:

https://github.com/Thoughtscript/java-refresh-notes/blob/master/src/io/thoughtscript/refresh/sealed/SealedExample.java

Java: Dynamic Proxying

Java Proxy Classes (Java Proxies):

Are encouraged in Clean Code.
Can be thought of as a Facade (a Wrapper with additional functionality to simplify use).
Often target some range of Interface implemenations.
Used to invoke or execute some additonal operation or imbue an operation with extra functionalities. (Additional logging, intercepting a method (via InvocationHandler), etc.)

Resources and Links

Java: Abstraction

Interfaces and Abstract Classes allow for the general structure of Implementing Classes to be planned out and further specified in those implementations.
Interfaces are lighter-weight and impose fewer suppositions on implementing Classes than Subclasses of Abstract Classes.
Abstract Classes typically contain fully-defined Methods that are then Overridden whereas Interfaces only contain Method signatures.
Both Interfaces and Abstract Classes can have fields defined within them that are borne by their Implementing or Subclassing Classes.

Interface

Interfaces support Multiple Inheritance (whereas Classes don't).
Methods defined in Interfaces are Public and Abstract by default. Typically, only their signature is defined.
Are Implemented or Extended with the implements or extends keywords, respectively.
Fields defined in an Interface are public static final by default.

public interface ExampleA {
    public void methodA();
    void methodB();
}

public interface ExampleB extends ExampleA {
    void methodC();
}

public class ExampleBImpl implements ExampleB {
    public void methodA() {
        System.out.println("I'm methodA");
    }

    public void methodB() {
        System.out.println("I'm methodB");
    }

    public void methodC() {
        System.out.println("I'm methodC");
    }
}

Functional Interfaces

Lambdas can also be used with Functional Interfaces (which provide the most amount of customization when using lambda expressions):

public class HelloWorld{

    @FunctionalInterface
    interface CheckStringInterface {
        boolean checkStringLambda(String s);
    }

    private static boolean checkString(String s) { 
        return ((s != null) && (s != "" ) && (Character.isUpperCase(s.charAt(0))));    
    }

    private static CheckStringInterface checkStringInstance = 
        (String s) -> ((s != null) && (s != "" ) && (Character.isUpperCase(s.charAt(0))));

    public static void main(String []args){
        System.out.println(checkStringInstance.checkStringLambda("Test"));
        System.out.println(checkStringInstance.checkStringLambda("Alpha"));
        System.out.println(checkString("test"));
    }
}

We observe how the Lambda expression allows us to implement the checkStringLambda() method as we see fit. The same parameters must be retained in a specific implementation but we can go wild with whatever we want right of the arrow:

public class HelloWorld{

    @FunctionalInterface
    interface CheckStringInterface {
        boolean checkStringLambda(String s);
    }

    private static CheckStringInterface checkStringInstanceOne = 
        (String s) -> ((s != null) && (s != "" ) && (Character.isUpperCase(s.charAt(0))));

    private static CheckStringInterface checkStringInstanceTwo = 
        (String s) -> (s != null);

    public static void main(String []args){
        System.out.println(checkStringInstanceOne.checkStringLambda("Test"));
        System.out.println(checkStringInstanceTwo.checkStringLambda("Alpha"));
    }
}

@FunctionalInterface must only be used on an Interface with one Method. A Functional Interface is an Interface with a single method to implement and allows Java to have a degree of Functional Programming in what is otherwise resolutely Object Oriented.

Abstract Classes

Abstract Classes are not themselves instantiated (although they can contain a Constructor) - they are Subclassed and instantiated thereby.
Methods belonging to an Abstract Class can be Abstract and they can be Overridden.
Abstract Classes are Subclassed using the extends keyword and their methods are Overridden using the @Override annotation.

public abstract class ExampleA {

    public void methodA() {
        System.out.println("I'm methodA from within ExampleA");
    }

    abstract void methodB();
}

public class ExampleB extends ExampleA {

    @Override
    public void methodA() {
        System.out.println("I'm methodA from within ExampleB");
    }

    public void methodB() {
        System.out.println("I'm methodB");
    }
}

Java: Singletons

A Singleton is a Class that's instantiated once and reused everywhere as that, single, copy.

Not Thread Safe

   // Sloppy not thread safe - HackerRank
   class Singleton{
      private static final Singleton instance = new Singleton();
      public static String str;

      private Singleton() {}

      public static Singleton getSingleInstance() {
        return instance;
      }
   }

Thread Safe Implementations

   import java.util.concurrent.Semaphore;

   // Refer to: https://www.digitalocean.com/community/tutorials/thread-safety-in-java-singleton-classes
   public class ThreadSafeSingleton {

      // Requires the CPU not to reorder when the variable is read using volatile.
      // Avoids a scenario where prior to initialization the variable might
      // be null in a secondary Thread.
      private static volatile ThreadSafeSingleton instance;

      // Private constructor for Singleton
      private ThreadSafeSingleton() {}

      public static ThreadSafeSingleton getOrCreateInstance() {
        // Separate out the volatile variable into another copy so no
        // copies are read by different threads here.
        ThreadSafeSingleton result = instance;

        if (result == null) {
              // synchronized
              synchronized (ThreadSafeSingleton.class) {
                result = instance;
                if (result == null) {
                    instance = result = new ThreadSafeSingleton();
                }
            }
        }
        return result;
      }

      // Mutex - initialize beforehand in a static context
      // Obviously, since a Singleton is only a single instance don't put Mutex's in a Main method or above as a field:
      // e.g. - private final Semaphore mutex = new Semaphore(1);
      public static ThreadSafeSingleton getOrCreateInstanceWithSemaphore(Semaphore mutex) throws InterruptedException {
        // Separate out the volatile variable into another copy so no
        // copies are read by different threads here.
        ThreadSafeSingleton result = instance;

        if (result == null) {
            if (mutex.availablePermits() > 0) {
                mutex.acquire();
                result = instance;
                if (result == null) {
                    instance = result = new ThreadSafeSingleton();
                }
                mutex.release();
            }
        }
        return result;
      }
   }

Resources and Links

Code samples:

https://github.com/Thoughtscript/java-refresh-notes/tree/master/src/io/thoughtscript/refresh/patterns

Java: Visibility and Access Modifiers

Access Modifiers

public - Everywhere within the Application, all Subclasses.
protected - Same Package, Subclasses regardless of Package.
package/none/default - Same Package, Subclasses in same Package.
private - Class and Object only, no Subclasses.

Static vs Nonstatic

static - belongs to the Class within which it resides.
non-static - belongs to the Object/instance and is checked at runtime.

A Non-static method belongs to the specific Object/instantiation of a particular Class. Non-static methods thus require an Object created via the constructor keyword new and are invoked directly from that created Object.

Final

The final keyword specifies that a variable is immutable, a constant.
The final keyword specifies that a method cannot be Overridden.
The final keyword specifies that a class cannot be Extended / Subclassed.

Sealed

Specifies that only certain Classes can inherit from or implement from the sealed Class or Interface.

An optional and potentially intermediate access visibility modifier-like keyword.
Specifies the exact Subclasses that can Subclass. (Even the Default/Package can be unduly permissive – consider Classes that have encryption hashes/ciphers/salts.)
Or, Classes that can implement an Interface.
sealed Class constraints:
- Permitted Subclasses must belong to the same module as the sealed Class.
- Every permitted Subclass must explicitly extend the sealed Class.
- Every permitted Subclass must define a modifier: final, sealed, or non-sealed.
Generally, a sealed-type hierarchy can have a Class or an Interface as its root.
- The remainder of the hierarchy can contain Classes or Interfaces, provided all leaf nodes of the hierarchy are either final concrete Classes or are non-sealed.
- If a leaf element is non-sealed, it can be either a Class or an Interface.

Example hierarchy from https://blogs.oracle.com/javamagazine/post/java-sealed-types-subtypes-final:

sealed interface Top permits A, B, C {}
non-sealed class A implements Top {}
record B(Top s) implements Top {}
enum C implements Top { SUB_A{}, SUB_B{} }

Style Guides

Resources and Links

Code samples:

Java: Checked and Unchecked Exceptions

Some Java Exceptions are Checked - they require a throws keyword or a try-catch block at Write Time, Compile Time.

Others aren't. They are Unchecked - handled and Thrown at Run Time.

Examples:

ArrayIndexOutOfBoundsException is Unchecked.
NullPointerException is Unchecked.
Parsing format exceptions are typically Checked. e.g. - ParseException

Handling

Interesting scenario:

class BatteryException extends Exception { }

class FuelException extends Exception { }

public class Car {
  public String checkBattery() throws BatteryException {
    // implementation
  }
  public String checkFuel() throws FuelException {
    // implementation
  }
  public String start() {
    try {
      checkBattery();
      checkFuel();
    } catch (BatteryException be) {
      return "BadBattery";
    } finally {
      return "";
    }
  }
}

One might be tempted to think the compiler refuses to compile since FuelException isn't caught. However, the presence of the finally clause overrules that typical requirement.
One might also be tempted to think that "BadBattery" is returned or FuelException is thrown from start(). Again, the presence of the finally clause overrules that incorrect but informed intution. (Regardless of the implementation of checkBattery() and checkFuel() the end result will always be the same.)

In the above scenario: start() will always return a "".

From: https://blogs.oracle.com/javamagazine/post/java-quiz-try-catch-finally-exception

Resources and Links

https://blogs.oracle.com/javamagazine/post/java-quiz-try-catch-finally-exception

Code samples:

https://github.com/Thoughtscript/java-refresh-notes/blob/master/src/io/thoughtscript/refresh/finallyblock/FinallyExample.java

Java: Errors

Fatal Errors at Runtime will terminate an Application killing the Process.

Exceptions, by contrast, are undesired or predicted defects in some code that are Thrown, Caught, and Handled at Compile Time and Run Time.

Finally

A finally block may be reached and executed when a fatal system Error occurs or if the Process is killed early (deliberately).

// JavaScript
process.exit()

// Java
System.exit(0);

Java: Arrays

Use length to access the size of the Array.

Java Arrays are fixed in their size after initialization.

Initialization

// With dimensions/size using this syntax.
// The default way
int[] intArrA = new int[8];
int[] intArrB;

Array initialization with values:

int[] intArrA = {1,2,3,4000,707,707,3,3};

Reassignment

Note that an Array A that's been assigned a size a > b can be reassigned with an Array value of size b without limitation.

int[] A = {1,2,3,4,5,6,7,8};
int[] B = {5,5,5,5,5};
A = B;

for (int i = 0; i < A.length; i++) {
    System.out.println(A[i]);
}

/*
5
5
5
5
5
*/

Thus, initializing an Array like so:

int[] C;
C = A;

for (int i = 0; i < C.length; i++) {
    System.out.println(C[i]);
}

/*
5
5
5
5
5
*/

can help to reduce invalidly declared Arrays.

Reference Types

Remember that primitive Arrays aren't Autoboxed to their reference types. (Also, that the type of an Array is the type of each element.) (Each individual element can be in the right circumstances, however.)

Consequently, int[] won't be Autoboxed to Integer[] nor Object[] and so doesn't meet the constraints of either an Object[] o parameter nor the generic <T>, T[].

public static <T> void printArray(T[] o) {
    for (int i = 0; i < o.length; i++) {
        System.out.println(o[i]);
    }
}

public static void main(String[] args) {
    // Remember that int[] isn't autoboxed to Integer[] 
    // only each int[i]

    // Also Object[] requires that int[] would be autoboxed to Integer[] first
    // So, either set that explicitly or convert
    Integer[] intArr = {1, 2, 3};
    String[] strArr = {"Hello", "World"};
    printArray(intArr);
    printArray(strArr);
}

To and From ArrayLists, Lists

To ArrayList:

Integer[] array = {1, 2, 3};

List<Integer> listA = List.of(array);

List<Integer> listB = Arrays.asList(array);

ArrayList<Integer> listC = new ArrayList<Integer>(Arrays.asList(array));

And back again:

// Will create an Array of length equal to the Collection size
Integer[] arrA = listA.toArray();

// Faster zero-size Array
// Better for Casting and Generics
// Use this even over .toArray(new Integer[listB.size()])
Integer[] arrB = listB.toArray(new Integer[0]);

Refer to: https://www.baeldung.com/java-collection-toarray-methods

Concatenation

Solutions that work for both Object and Primitive Type Arrays:

int[] arrA = {1, 2, 3};
int[] arrB = {4, 5, 6};
int[] result = new int[arrA.length+arrB.length];

for(int i = 0; i < arrA.length; i++) {
    result[i] = arr1[i];
}

for(int i = 0;i < arrB.length; i++) {
    result[arrA.length + i] = arrB[i];
}

java.util.Arrays - copyOf():

import java.util.Arrays;

//....
int[] arrA = {1, 2, 3};
int[] arrB = {4, 5, 6};
int[] result = Arrays.copyOf(arrA, arrA.length + arrB.length);

System.arraycopy(arrB, 0, result, arrA.length, arrB.length);

org.apache.commons.lang.ArrayUtils - addAll():

import org.apache.commons.lang.ArrayUtils;

int[] arrA = {1, 2, 3};
int[] arrB = {4, 5, 6};

int[] result = ArrayUtils.addAll(arrA, arrB);

Sorting

Refer to the article on Comparisons.

int[] arrA = {5, 6, 3, 9, 4, 5, 6,1, 2, 3};

// Natural sort
Arrays.sort(arrA);

parallelSort() is performant at element sizes greater than 1,000,000.

Otherwise, it's likely less performant than a standard sort().

Arrays.parallelSort(arrA);

Resources and Links

Code samples:

Java: Collections

Remember that Collections require Reference Types not Primitive Types: e.g. Set<Integer> vs. Set<int>.

List

List<String> list = new ArrayList<String>();
    list.add("A");
    list.add("B");
    list.add("C");

indexOf() is a List method not available on Arrays (without conversion).
Use LinkedList for more expedient modifications.

// Conversion between int array to List

// Reference type from primitive - either stream or use reference type
Integer[] intArrA = {1,2,3,4000,707,707,3,3};

// Must use reference type 
List<Integer> intList = Arrays.asList(intArrA);

// Or to ArrayList
ArrayList<Integer> listB = new ArrayList<Integer>(Arrays.asList(intArr));

UnmodifiableList

List<String> listC = new ArrayList<>(Arrays.asList("one", "two", "three"));
List<String> unmodifiableList = Collections.unmodifiableList(listC);
// unmodifiableList.add("four"); // unmodifiableList is immutable!

// (if listC is modified so will the view unmodifiableList - e.g:
// listC.add("four");)

For more:

Set

Set<Integer> setC = new HashSet<>();
    setC.add(0);
    setC.add(1);
    setC.add(2);
    setC.add(3);
    setC.add(4);

bool x = setC.contains(0);

Map

// ...<Key, Value>
Map<String, String> hmA = new HashMap<String, String>();
    hmA.put("0", "A");
    hmA.get("0"); // null or Value
    hm.containsKey("0"); // true if Key is present, false otherwise

HashMap

Java Collections HashMap is a basic implementation of Map.
It uses hashCode() to estimate the position of a Value and to create hashed Keys (using backing buckets to store hashes in a Hash Table). The number of buckets used is called the capacity (not to be confused with the number of Keys or Values).
It accepts null as a Key and isn't Thread Safe.
Keep the Key range small to avoid unnecessarily large buckets in-memory.
Generally, Keys should be immutable. If a Key-Value pair requires alteration, evict and remove the previous Key and create a new entry.

Note that a HashSet has Objects as Values that are hashed into Keys using a backing HashMap. This guarantees uniqueness/deduplication.

Resources and Links

Code samples:

Java: Streams

Streams are:

Immutable

Used once.

import java.util.stream.Collectors;
import java.util.stream.Stream;

public class Example {
   public static void main(String args[]) {
      Stream<Integer> myStream = Stream.of(1,2,3,4,8,7,6,5,4);
      myStream.sorted((a, b) -> a - b);
      myStream.map(x -> x + 1).collect(Collectors.toList());
   }
}

Exception in thread "main" java.lang.IllegalStateException: stream has already been operated upon or closed
//...

List

From List:

List<Integer> myList = new ArrayList<Integer>();

for(int i = 1; i< 10; i++){
    myList.add(i);
}

Stream<Integer> myStream = myList.stream();

To List:

Stream<Integer> myStream = //...

// Faster way to do this: myStream.collect(Collectors.toList());
List<Integer> myList = myStream.toList();

Array

From Array:

Integer[] myArr = {1, 2, 3, 4, 5};

Stream<Integer> myStream = Arrays.stream(myArr);

To Array:

Stream<Integer> myStream = //...

Integer[] myArr = myStream.toArray(Integer[]::new);

Examples

Sort using sorted to return the third lowest Example id - required to use a Stream.

Given:

public class Example {
    private int id;
    private String name;

    // Constructor
    //... Getters and Setters
}

Example One

Stream<Example> myStream = //...

// Implements a Comparator
Stream<Example> sorted = myStream.sorted((a, b) -> a.getId() - b.getId());

// toList() via Collector
List<Example> myList = sorted 
  .collect(Collectors.toList());

myList.get(3);

Example Two

From a List:

List<Example> myList = //...

// Don't modify myList in place
Integer[] myArr = myList.toArray(new Integer[0]);

Stream<Example> myStream = Arrays.stream(myArr);

// Implements a Comparator
Stream<Example> sorted = myStream.sorted((a, b) -> a.getId() - b.getId());

// toList() via Collector
List<Example> myList = sorted 
  .collect(Collectors.toList());

myList.get(3);

Example Three

From a List working with result Array:

List<Example> myList = //...

// Don't modify myList in place
Integer[] myArr = myList.toArray(new Integer[0]);

Stream<Example> myStream = Arrays.stream(myArr);

// Implements a Comparator
Stream<Example> sorted = myStream.sorted((a, b) -> a.getId() - b.getId());

Integer[] myArr = myStream.toArray(Integer[]::new);

myArr[3];

Parallel Streams

Use when order doesn't matter.

Leverages underlying worker pooling from the Common Worker Pool:

List<Integer> myList = Arrays.asList(1, 2, 3, 4);

myList.parallelStream().forEach(num ->
    System.out.println(myList + " " + Thread.currentThread().getName())
);

Common Operations

forEach

list.stream().forEach(consumer);

Stream<Example> myStream = //...
myStream.forEach(System.out::print);
myStream.forEach(x -> System.out.println(x));

map

list.stream().map(x -> x + 1);

Stream<Example> myStream = //...
myStream.map(x -> x + 1);

Reference Type to Primitive Type

Use IntStream, LongStream, or DoubleStream for any primitive numeric typed Stream.

Integer[] myArr = {1, 2, 3, 4, 5};

Stream<Integer> myStream = Arrays.stream(myArr);

IntStream is = s.mapToInt(i -> i);

Eliding

Streams will occasionally omit certain internal operations if:

Doing so does not modify the outcome or result of the computation.
The Stream is passed into some kind of dynamic processing where the size or count cannot be automatically determined beforehand.

Logging and output messages, in particular, will occasionally fail to display.

Refer to:

Resources and Links

Code samples:

Java: Primitive Types

Each Primitive Type has a corresponding Reference Type which wraps the underlying Primitive Type with additional functionality. Furthermore, Reference Types allow null values where they wouldn't be typically allowed (e.g. an int has no null value under normal circumstances - its default value is 0).

Note: Strings are essentially Reference Types with char as their corresponding Primitive Type. Note further: char has Character as its correspond Reference Types.

int

int types must be instantiated with a value anywhere but a field in or on a Class itself:

public class Metrics {
    private static int total;

    public static void main(String args[]) {
        System.out.println(total);
    }
}

Otherwise, initialize the variable with a value:

public class Metrics {    
    public static void main(String args[]) {
        int total = 0;
        System.out.println(total);
    }
}

Note that int can't support precise fractional division without rounding, using a String, or some LCD algo (Least Common Denominator).

Java: Strings, StringBuffer, StringBuilder

Java Strings use the String Pool, are immutable, and are Primitive Data Types.
StringBuilder does not use Java's String Pool, isn't Thread Safe, and is better (performance and memory-wise) for concatenating complex and lengthy Strings. (StringBuilder is used automatically under the hood now for simple String concatenations like String c = a + b + ":".)
StringBuffer is the Thread Safe version of StringBuilder.

String Pool

Java's String Pool points Strings with the same value to the same item in-memory.

This optimization feature is designed to offset some performance costs associated with the immutability of Java's Strings and is referred to as Interning.

Interning can be manually set via String.intern():

String x = new String("abc");
String y = x.intern();
// Will be the same
// Useful when using a String constructor

String Constructor

Using a Constructor will force two Strings with the same text to refer to different items in memory.

Caveat: unless -XX:+UseStringDeduplication is enforced at the JVM level (say, as an arg) or if intern() isn't called.

String a = "abc";
String b = new String("abc");
// Not the same memory address

String c = "abc";
// Strings a and c have the same text value and the same memory address

Comparison

Remember the following:

String x = "abc";
String y = "abc";
System.out.println(x == y); // true

String z = new String("abc");
System.out.println(x == z); // false

JVM Fine-Tuning

-XX:StringTableSize=4901

The maximum StringTableSize Java 11+ is now 65536.

Note: the -X and -XX affixes indicate non-standard and potentially unstable "power user" JVM parameters settings, respectively.

StringBuilder

Using + in a (large) loop will still (in Java 18) result in wonkiness:

Inefficient from a time-complexity standpoint.
The underlying implementation (automatic conversion between + and StringBuilder) is imperfectly performant for large cases.
Will actually be indeterministic (as I found out). And interestingly in at least two ways:
- By machine.
- With identical code blocks being run sequentially.

Consider generating a String hash for comparing Sets of Arrays:

Set<int[]> exampleA = new HashSet<>();
exampleA.add(new int[]{1, 2});
exampleA.add(new int[]{3, 4});
exampleA.add(new int[]{5, 6, 7});

String stringHashA = "";

for (int[] entry : exampleA) {
    stringHashA += "{";
    for (int i = 0; i < entry.length; i++) {
        stringHashA += entry[i];
        if (i < entry.length - 1) stringHashA += ",";
    }
    stringHashA += "}";
}

Set<int[]> exampleB = new HashSet<>();
exampleB.add(new int[]{1, 2});
exampleB.add(new int[]{3, 4});
exampleB.add(new int[]{5, 6, 7});

String stringHashB = "";

for (int[] entry : exampleB) {
    stringHashB += "{";
    for (int i = 0; i < entry.length; i++) {
        stringHashB += entry[i];
        if (i < entry.length - 1) stringHashB += ",";
    }
    stringHashB += "}";
}

System.out.println(stringHashA);
System.out.println(stringHashB);
System.out.println(stringHashA.equals(stringHashB));
// stringHash will be indeterministic with respect to the two identical code blocks
/*
    {5,6,7}{3,4}{1,2} - this is deterministic and will always print this on same machine
    {5,6,7}{1,2}{3,4} - this is deterministic and will always print this on same machine
    false
*/
/*
    {3,4}{1,2}{5,6,7} - on a different machine
    {5,6,7}{1,2}{3,4} - on a different machine
    false
*/

Set<int[]> exampleA = new HashSet<>();
exampleA.add(new int[]{1, 2});
exampleA.add(new int[]{3, 4});
exampleA.add(new int[]{5, 6, 7});

StringBuilder stringHashA = new StringBuilder();
for (int[] entry : exampleA) {
    stringHashA.append("{");
    for (int i = 0; i < entry.length; i++) {
        stringHashA.append(entry[i]);
        if (i < entry.length - 1) stringHashA.append(",");
    }
    stringHashA.append("}");
}

Set<int[]> exampleB = new HashSet<>();
exampleB.add(new int[]{1, 2});
exampleB.add(new int[]{3, 4});
exampleB.add(new int[]{5, 6, 7});

StringBuilder stringHashB = new StringBuilder();
for (int[] entry : exampleB) {
    stringHashB.append("{");
    for (int i = 0; i < entry.length; i++) {
        stringHashB.append(entry[i]);
        if (i < entry.length - 1) stringHashB.append(",");
    }
    stringHashB.append("}");
}

System.out.println(stringHashA.toString());
System.out.println(stringHashB.toString());
System.out.println(stringHashA.toString().equals(stringHashB.toString()));
// stringHash will be indeterministic with respect to the two identical code blocks
/*
    {5,6,7}{3,4}{1,2} - this is deterministic and will always print this on same machine
    {3,4}{1,2}{5,6,7} - this is deterministic and will always print this on same machine
    false
*/
/* 
    {3,4}{1,2}{5,6,7} - on a different machine
    {1,2}{3,4}{5,6,7} - on a different machine
    false
*/

However:

Set<int[]> result = new HashSet<>();
result.add(new int[]{1, 2});
result.add(new int[]{3, 4});
result.add(new int[]{5, 6, 7});

StringBuilder stringHashResult = new StringBuilder();
for (int[] entry : result) {
    stringHashResult.append("{");
    for (int i = 0; i < entry.length; i++) {
        stringHashResult.append(entry[i]);
        if (i < entry.length - 1) stringHashResult.append(",");
    }
    stringHashResult.append("}");
}

System.out.println(resultStringHash.toString());
// stringHash will be deterministic on same machine but will generate different results otherwise

Refer to: https://stackoverflow.com/questions/11942368/why-use-stringbuilder-explicitly-if-the-compiler-converts-string-concatenation-t

Resources and Links

Java: Important Keywords

final - can't be changed as a variable, can't be Overridden as a method, and can't be subclassed as a Class.
static
- On a method: belongs to the Class not an instance or Object.
- On a Class Field: the Value is shared between all Instances (akin to the @@ Class Variable in Ruby.)
- static block allows for multiline initializations of static Class Variables.
sealed - specifies that only certain Classes can inherit from or implement from the sealed Class or Interface.
abstract - on a Class: can extend but not instantiate, can Override methods of the inherited Abstract Class.
synchronized - enforce thread safety within the specified Object or Class.
transient - ignore value, set a default value, and save or read from a serialized file.
record - syntactic sugar to specify an immutable POJO / DTO.
volatile - ignores CPU and memory caching and allows the actual CPU instruction order to be preserved. (Due to the nature of multi-threaded CPU's, instructions, values, etc. can be incorrectly handled by the CPU cache. In those circumstances it can be beneficial to side-step caching entirely.)

Resources and Links

Code Samples:

Java: Security

Java String over Character Array?

Strings remain in Heap for an unspecified amount of time in between Garbage Collecting. As such, using a Character Array can be a better choice since one can alter the contents of each index of the Array at any time between Garbage Collection events.

So, it can be a great choice for sensitive PII data that's in-memory.

Transient Serialization

When one uses the transient keyword, default values are Persisted and the original values are stored in a separate file. (This was often used for PII since it separates sensitive data into two parts that have to be reassembled.)

Scanners and Tools

Resources and Links

Java: Generics

Generics abstract or generalize a method so that the specific details of the argument types are not required.

By convention:

/**
    Type Parameters:
    E - Element (used extensively by the Java Collections Framework)
    K - Key
    N - Number
    T - Type
    V - Value
    S,U,V etc. - 2nd, 3rd, 4th types
*/

Type Generics

public class KeyedTreeMap<T> implements Map<KeyedTreeMap<T>> {
    //...
}

Type Generics apply to Reference Types, not Primitive Types.

Bounded Type Parameters

We can require that a generic be constrained to being a certain type:

public static <X extends Integer> void exampleMethod(X[] genericArr) {
    //...
}

public interface Pair<K, V> {
    public K getKey();
    public V getValue();
}

public class ExamplePair<K, V> implements Pair<K, V> {
    private K key;
    private V value;

    public ExamplePair(K key, V value) {
        this.key = key;
        this.value = value;
    }

    public K getKey()    { return key; }
    public V getValue() { return value; }
}

//Instantiations of the ExamplePair class:
ExamplePair<String, Integer> p1 = new ExamplePair<String, Integer>("Even", 8);
ExamplePair<String, String>  p2 = new ExamplePair<String, String>("hello", "world");

Java: Techniques

Algo Quick Reference

Common operations specific to algorithm implementation:

// length, length(), size()
myArray.length;
myString.length();
myList.size();

// List
myListA.addAll(myListB); // Merge two Lists
myListA.add(0, myObject); // Prepend, unshift - adds an element to the beginning
myListA.add(myIdx, myObject); // Adds an element at index
myListA.pop(); // Pop - removes last element and returns
myListA.remove(myIdx); // Remove element at index
myListA.sort((a,b) -> b - a); // Custom sort with Lambda Comparator

// Conversion
List<Integer> listA = List.of(myArray); // Convert Array to List
List<Integer> listB = Arrays.asList(myArray); // Convert Array to List
Integer[] arrA = listA.toArray(); // Convert List to Array

// Maps
myMap.put(myKey, myValue); // Add Key Value pair to Map
myMap.containsKey(myKey); // Check if Key exists in Map without traversal
myMap.get(myKey); // Get Value using Key - Null if not found

// Chars
Character.isDigit(myChar); // Check if char is 0-9
Character.isLetter(myChar); // Check if char is letter
Character.isLetterOrDigit(myChar);  // Check if char is letter or 0-9
Character.isUpperCase(myChar); // Check if is uppercase
char myChar = myString.charAt(myIdx); // Get char at idx of String
String myStr = String.valueOf(myChar); // char to String
String myStr = new String(new char[]{'a', 'b', 'c'}); // chars to String

// Strings
String[] start = myString.split(""); // Split String
myString.substring(inclusiveIdx, exclusiveIdx); // Substring
myString.contains(otherString); // Substring match within

// Arrays
int[][] deepCopy = Arrays.copyOf(myArray, myArray.length); // Deep copy an Array
Arrays.sort(myArray); // Sort ascending
Arrays.sort(myArray, Collections.reverseOrder()); // Sort descending
Arrays.sort(myArray, (a, b) -> b - a); // Custom sort with Lambda Comparator

// Reference to Primitive Type
myInteger.intValue(); // Get int from Integer
myCharacter.charValue(); // Get char from Character

// int
int M = heap.size() / 2; // Same as int M = (int) (heap.size() / 2); which is the same as calling Math.foor(...) to drop decimals values.

String to Date

SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd");
Date nextDate = formatter.parse("2024-01-25");

Date to Calendar

Useful to know this conversion since Calendar supports basic day and week addition operations.

SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd");
Date nextDate = formatter.parse("2024-01-25");
Calendar c = Calendar.getInstance();
c.setTime(nextDate);

char to int

Remember that merely casting a char to an int (recommended/encouraged by IntelliJ) will actually produce the wrong conversion:

String strNum = "12345";
char c = strNum.charAt(0);
int cc = (int) c;
int ccc = Character.getNumericValue(c);
System.out.println(cc == ccc); // false

Note that Integer.parseInt() will also accomplish the correct conversion along with Character.getNumericValue():

int ccc = Character.getNumericValue(c);
int cccc = Integer.parseInt(String.valueOf(c));
System.out.println(ccc == cccc); // true

length, size, and count

Check specific Character in String:

myArray.length;
myString.length();
myList.size();
myStream.count();

array slicing

String[] L = new String[] { "Python", "Java", "Kotlin", "Scala", "Ruby", "Go", "Rust" };
String[] R = Arrays.copyOfRange(L, 1, 4);

Refer to: https://www.baeldung.com/java-slicing-arrays

chars in String

Check specific Character in String:

char c = str.charAt(0);
String s = Character.toString(c);
boolean beginClosed = s.equals("[");

Sort an Array

2D:

int[][] intervals = [[1,2],[3,4],[5,6],[7,8]];

Arrays.sort(intervals, (a,b) -> {
    if (a[0] < b[0]) return -1;
    if (b[0] < a[0]) return 1;
    return 0;
});

1D:

int[] nums = [6,5,4,3,8,1];

Arrays.sort(nums);

Swap List Indicies

int l = 0;
int r = 5;

while (l < r) {
    Collections.swap(listInt, l, r);
    l++;
    r--;
}

Java Mail

To configure, install, and use Java Mail:

<!-- pom.xml -->
<!-- Java Email Dependencies -->
<dependency>
    <groupId>com.sun.mail</groupId>
    <artifactId>javax.mail</artifactId>
    <version>1.5.5</version>
</dependency>

# application.yml
spring:
  # Enable auth
  mail:
    host: smtp.gmail.com
    port: 587
    username: aaaaaaa
    password: aaaaaaa
    properties:
      mail:
        debug: true
        transport:
          protocol: smtp
        smtp:
          auth: true
          starttls:
            enable: true

package io.thoughtscript.example.services;

import io.thoughtscript.example.Constants;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.mail.SimpleMailMessage;
import org.springframework.mail.javamail.JavaMailSender;
import org.springframework.stereotype.Service;

@Slf4j
@Service
public class EmailService {

    @Autowired
    JavaMailSender javaMailSender;

    public void sendMagicEmail(String emailAddress, String token) {
        SimpleMailMessage email = new SimpleMailMessage();

        StringBuffer emailContent = new StringBuffer();
        emailContent.append(Constants.EMAIL_MAGIC_LINK_GREETING);
        emailContent.append(Constants.AUTH_LOGIN_ENDPOINT_FULLY_QUALIFIED);
        emailContent.append("?token=");
        emailContent.append(token);
        emailContent.append("&email=");
        emailContent.append(emailAddress);

        email.setFrom("testapp@test.com");
        email.setTo(emailAddress);
        email.setText(emailContent.toString());
        email.setSubject("test");

        log.info(email.getTo()[0]);
        log.info(email.getText());
        log.info(email.getSubject());

        javaMailSender.send(email);
    }
}

Refer to: https://www.digitalocean.com/community/tutorials/javamail-example-send-mail-in-java-smtp

Resources and Links

Java: Asynchronous Programming

Java Threads

Java Threading is mapped to underlying System Threads by default (logical or physical CPU cores).
Java Threads can be pooled to help ensure that the application's needs don't overwhelm the available physical resources.
Java Thread Pooling relies heavily on the Executors and Runnable implementations:
- Runnable - implementations of Runnable can be passed into Executors to be run automatically.
- ExecutorService - provides factories for defining Thread Pool sizes on the fly or programmatically.
- ThreadPoolExecutor - use for fine tuning a Thread Pool.
- ScheduledExecutorService - for executing a task at a specific time.

Examples:

ExecutorService executor = Executors.newFixedThreadPool(10);

ExecutorService nonBlockingService = Executors.newSingleThreadExecutor();
nonBlockingService.execute(() -> {
    jmsClient.sendObject();
});

public class WorkerThread implements Runnable {
    @Override
    public void run() {//...}
}

ThreadPoolExecutor executorPool = new ThreadPoolExecutor(//...);
executorPool.execute(new WorkerThread());

Refer to: https://www.baeldung.com/java-executor-service-tutorial and https://www.javatpoint.com/executor-framework-java

Futures vs CompletableFutures

Futures
- Blocking.
- Can't combine results from more than one Thread.
CompletableFutures
- Added in Java 8 and significantly improved in Java 9.
- Chainable methods allow for JavaScript Promise-like resolve(), reject(), and promiseAll() behavior.
- Can combine results from multiple Threads.
- Non-blocking.

Asynchronous Promises

Resolving and using an Asynchronous Objecct: CompletableFuture.supplyAsync(() -> { //... }).thenApplyAsync(result -> { //... })

if (openCsvService.validate(file)) {
    CsvTransfer csvTransfer = openCsvService.setPath(file);
    Instant start = TimeHelper.start(Constants.ALL);

    return CompletableFuture.supplyAsync(() -> {
        try {
            CompletableFuture<String> save = openCsvService.saveCsv(file);
            responseTransfer.setMessage(save.get());
        } catch (Exception e) {
            responseTransfer.setMessage(CSV_UPLOAD);
        }
            return responseTransfer;
        }).thenApplyAsync(result -> {
            try {
                CompletableFuture<List<String[]>> allPromise = openCsvService.readCsvAll(csvTransfer);
                result.setCsv(allPromise.get());
            } catch (Exception e) {
                responseTransfer.setMessage(GENERIC_EXCEPTION);
            }
            result.setPerformance(TimeHelper.stop(start));
             return result;
        });
    }

    responseTransfer.setMessage(CSV_VALIDATION_EXCEPTION);
    return CompletableFuture.completedFuture(responseTransfer);
}

Thread Safety

synchronized - enforce thread safety within the contents of a Class or Object.
- On a Static method, thread safety is enforced on the Class.
- On a Non-Static method, thread safety is enforced on the Object.
Atomic Objects have inherently Synchronized values.
Refer to the article on Thread-Safe Singletons.
Refer to the article on Computer Science Processes and Threads.
Java provides Semaphores and Locks:
- ReentrantLocks allow a single Thread to (re)access a resource at a time.
- Semaphores allow a specified number of Threads to share access to a resource at any given time.
Wait() vs Sleep():
- Wait() - belongs to Object, Non-Static, should only be called from a Synchronized context.
- Sleep() - belongs to Thread, Static,

Resources and Links

Java: SOAP

Producer

Generates Classes from an .xsd file.
Provides Web Services via the @EnableWs configuration annotation and @Endpoint handler annotation.
Makes a WSDL accessible to SOAP clients so relevant Classes can be generated client-side.

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema"
        targetNamespace="/jaxb/gen"
        xmlns:examples="/jaxb/gen"
        elementFormDefault="qualified">

    <element name="getComplexExampleRequest" type="examples:GetComplexExampleRequest"></element>
    <element name="getComplexExampleResponse" type="examples:GetComplexExampleResponse"></element>
    <element name="complexExample" type="examples:ComplexExample"></element>
    <element name="simpleStringEnumExample" type="examples:SimpleStringEnumExample"></element>

    <complexType name="GetComplexExampleRequest">
        <sequence>
            <element name="exampleId" type="int"/>
        </sequence>
    </complexType>

    <complexType name="GetComplexExampleResponse">
        <sequence>
            <element name="complexExample" type="examples:ComplexExample"/>
            <element name="name" type="string"/>
            <element name="gender" type="string"/>
            <element name="created" type="dateTime"/>
        </sequence>
    </complexType>

    <complexType name="ComplexExample">
        <sequence>
            <element name="exampleId" type="int"/>
            <element name="description" type="string"/>
            <element name="stringEnum" type="examples:SimpleStringEnumExample"/>
        </sequence>
    </complexType>

    <simpleType name="SimpleStringEnumExample">
        <restriction base="string">
            <enumeration value="HELLO"/>
            <enumeration value="SUCCESS"/>
            <enumeration value="FAIL"/>
        </restriction>
    </simpleType>
</schema>

@EnableWs
@Configuration
public class SoapConfig extends WsConfigurerAdapter {

    @Bean
    public ServletRegistrationBean messageDispatcherServlet(ApplicationContext applicationContext) {
        MessageDispatcherServlet servlet = new MessageDispatcherServlet();
        servlet.setApplicationContext(applicationContext);
        servlet.setTransformWsdlLocations(true);
        return new ServletRegistrationBean(servlet, "/ws/*");
    }

    @Bean(name = "complexExample")
    public DefaultWsdl11Definition defaultWsdl11Definition(XsdSchema exampleSchema) {
        DefaultWsdl11Definition wsdl11Definition = new DefaultWsdl11Definition();
        wsdl11Definition.setPortTypeName("ComplexExamplePort");
        wsdl11Definition.setLocationUri("/ws");
        wsdl11Definition.setTargetNamespace(URI);
        wsdl11Definition.setSchema(exampleSchema);
        return wsdl11Definition;
    }

    @Bean
    public XsdSchema exampleSchema() {
        return new SimpleXsdSchema(new ClassPathResource(XSD));
    }
}

@Slf4j
@Endpoint
public class ComplexExampleEndpoint {

    @Autowired
    ComplexExampleRepository complexExampleRepository;

    @PayloadRoot(namespace = URI, localPart = "getComplexExampleRequest")
    @ResponsePayload
    public GetComplexExampleResponse getCountry(@RequestPayload GetComplexExampleRequest request) {
        GetComplexExampleResponse response = new GetComplexExampleResponse();
        response.setComplexExample(complexExampleRepository.findExample(request.getExampleId()));
        return response;
    }
}

Consumer

Generates Classes client-side by accessing the hosted WSDL.
Invokes calls against the Web Service using the specified WSDL Class schemas.

curl --header "content-type: text/xml" -d @curl-request.xml http://localhost:8080/ws

<!-- curl-request.xml -->
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:examples="https://github.com/Thoughtscript/java_soap_wsdl_2023">
   <soapenv:Header/>
   <soapenv:Body>
      <examples:getComplexExampleRequest>
         <examples:exampleId>1</examples:exampleId>
      </examples:getComplexExampleRequest>
   </soapenv:Body>
</soapenv:Envelope>

Resources and Links

Code samples:

https://github.com/Thoughtscript/java_soap_wsdl_2023

Java: Text Editors

Troubleshooting Major-Minor Versions

Set the correct SDK
- File > Project Structure > Project Settings > Project > SDK
- File > Project Structure > Project Settings > Project > Language Level
Check the Platform Settings
- File > Project Structure > Platform Settings > SDKs
Check the pom.xml
- <properties>
- <maven.compiler.source>
- <maven.compiler.target>
Check the Run and Debug Configurations
- In the Upper-Right Corner
- Select the Run/Debug Configurations dropdown > Edit Configurations... > Build and run

Spring: General Concepts

Inversion of Control - preconfigured default values reflecting best practices and commonly used patterns.

Resources and Links

Spring: Project Layout

Typical Project Layout

+- gradle
|  +- ...
|
+- build
|  +- ...
|
+- src
|  +- main
|  |  +- java
|  |  |  +- com
|  |  |     +- example
|  |  |        +- app
|  |  |           +- ...
|  |  |           +- Main.java
|  |  |
|  |  +- resources
|  |  |  +- application.properties
|  |  |  +- ...
|  |  |
|  |  +- webapp
|  |     +- resources
|  |     |  +- scripts
|  |     |     +- ...
|  |     |  +- styles
|  |     |     +- ...
|  |     +- WEB-INF
|  |     |  +- JSP
|  |     |     +- ...
|  |
|  +- test
|     +- java
|        +- com
|           +- example
|              +- app
|                 +- MainTest.java
|   
+- target
|  +- ...
|
+- pom.xml
+- gradlew.bat
+- build.gradle
+- settings.gradle 
+- .gitignore
+- README.md
+- ...

Common Maven Commands

mvn clean
mvn install
mvn spring-boot:run

Comman Gradle Commands

./gradlew clean
./gradlew build
./gradlew run

Java SSL Keytool

keytool -genkey \
  -alias interviewhelper \
  -keystore interviewhelper.p12 \
  -storetype PKCS12 \
  -keyalg RSA \
  -storepass F#4%afdfdsdfdfa*adf \
  -validity 730 \
  -keysize 4096

IntelliJ

Looks like .war artifacts don't populate immediately when a pom.xml has successfully been loaded into IntelliJ anymore.

Click File > Plugins > Type in Maven Helper
The Maven Helper plugin provides additional context menu options that appear to be missing now out of the box.
To correctly build a .war file, right-click on pom.xml > Run Maven > Plugins > maven-war-plugin > war:war

Resources and Links

Spring: Logging

Slf4j - The Simple Logging Facade for Java serves as an abstraction or interface for an underlying target logging framework or library.
Log4j - Apache, the original default logging library.
Logback - Successor to Log4j.
Log4j 2 - Apache's upgrade for Log4j that provides significant improvements over its predecessor and applies many lessons from Logback.
Lombok - A utility library that provides many helpful annotations. Includes Slf4j logging as an annotation.

Common Combinations

Lombok + Slf4j (included in Lombok) + Logback (included in Spring Boot Starters, the default)
Slf4j + Log4j

Resources and Links

Code samples:

https://github.com/Thoughtscript/spring_boot_2023 (Spring Boot + Lombok + Slf4j + Logback)

Spring: Annotations

Common Spring-related Decorator-style annotations.

Spring MVC

@PathVariable with @GetMapping("/{product_id}") to specify a URL Path Variable
@RequestBody - the HTTP Request Body data. Can be Form Data. By Key-Value.
@RequestParam - an HTTP Key-Value Request Paramater passed the URL string.
@RestController - annotates @Controller and @ResponseBody.

Jackson, JAX

@JsonNaming - allows snake case and camel case, to be used in a deserialized/serialized field.
@JsonPropertyOrder - specifies the exact serialization/deserialization order to be specified (since Spring will use alphabetical order by default). Sometimes a property must be computed after another regardless of alphabetical order.
@JsonIgnore - specifies that a field should be ignored during serialization. Useful for preventing infinite JSON recursion with One-to-Many, Many-to-Many, and Many-to-One table relationships.
@Transient - no to be confused with the transient keyword, @Transient specifies that a field should be ignored but does not involve Serialization.

Spring

@ComponentScan - specifies that the Application should recursively search for relevant Beans within the specified Package or Classpath.
@SpringBootApplication - annotates @Configuration, @EnableAutoConfiguration, and @ComponentScan.
@Configuration - specifies that the Bean is for configuration, contains configuration settings, Extends or Overrides some default configuration Bean, loads Application Properties, or sets them programmatically.
@EnableWebMvc - Spring (but not Spring Boot) configuration annotation.
@EnableAutoConfiguration - use the default auto-configured settings.
@EnableAsync and @Async - enables Asynchronous programming in Spring and configures the desired Thread Executor arrangment so the @Async annotation can be added to a Bean method automatically wrapping it with an Executor. Typically used with CompletableFutures and/or Futures.
@Transactional - specify that a method should be wrapped in database Transaction.
@EnableRetry, @Retryable, and @Recover - enable specific method invocations to be attempted multiple times (default of three) through method intercepting in the event of a specified Exception. Remember that @Recover is used to handle Exceptions emanating from an @Retryable attempt.

Java EE

@Entity - Javax Persistence annotation specifying that the POJO in question is relevant to Object Relational Mapping.
@Bean - Java EE annotation indicating that the Class is a Bean (should be initialized and used as such) within some configuration Class.

Resources and Links

Spring: Hibernate

The Hibernate Naming Strategy divides into two basic strategies: (1) ImplicitNamingStrategy and (2) DefaultNamingStrategy. The default naming strategy can be configured in application.properties or as a configured setting. (It is not set in the Hibernate SQL Dialect.)
Used with Spring Data, JPA, Entity Framework, and Javax Persistence.
Object Relational Mapping framework for converting SQL data into valid Java entities in-memory at runtime.

Relationships

Relations between data structures, Rows, and Tables are specified with a mix of annotations and data-layer constraints.

Note: _fk is sometimes appended to a column below but should not be confused with a Foreign Key (Foreign Key Constraint) proper. I use that convention to make it easy to read what a column is doing - it stands for foreign-key-like or what's sometimes referred as a Foreign Keys Without a Constraint. (Best practices encourage the use of true Foreign Key Constraints in Production, DRY JOIN tables, and removing any additional Foreign Key columns.) Consult: https://ardalis.com/related-data-without-foreign-keys/ for more on the nomenclature.

Lazy Loading

Note that fetch defaults to FetchType.EAGER (associated entities are loaded immediately). FetchType.LAZY will load those entities only when the field is first used.

Generally, Hibernate encourages FetchType.LAZY: "If you forget to JOIN FETCH all EAGER associations, Hibernate is going to issue a secondary select for each and every one of those which, in turn, can lead to N+1 query issues. For this reason, you should prefer LAZY associations."

Refer to: https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#architecture and https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#fetching

One to One

Example: User and UserProfile.

@PrimaryKeyJoinColumn

@Entity
@Table(name = "A")
public class A {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  //  If a Foreign Key is explicitly defined between A to B.
  @OneToOne(cascade = CascadeType.MERGE)
  @PrimaryKeyJoinColumn
  private B b;
}

@Entity
@Table(name = "B")
public class B {

  // @GeneratedValue
  @Id
  @Column(name = "id")
  private int id;
}

@JoinColumn

@Entity
@Table(name = "A")
public class A {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  @OneToOne(fetch = FetchType.EAGER)
  @MapsId
  @JoinColumn(name = "bId")
  private B b;
}

@Entity
@Table(name = "B")
public class B {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  // Optional
  @OneToOne(mappedBy = "b", cascade = CascadeType.ALL)
  private A a;
}

DROP TABLE IF EXISTS A;
CREATE TABLE A (
  id INT(11) NOT NULL AUTO_INCREMENT,
  bId INT(11) NOT NULL
  PRIMARY KEY (id)
);


DROP TABLE IF EXISTS B;
CREATE TABLE B (
  id INT(11) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (id)
);

One To Many

Example: Owner and Pet (there are many Pets that can be owned by an Owner). And, below many A are mapped to a single B.

@Entity
@Table(name = "A")
public class A {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;
}

@Entity
@Table(name = "B")
public class B {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  @OneToMany(cascade = CascadeType.ALL)
  @JoinTable(name = "B_A", joinColumns = { @JoinColumn(name = "bId") }, inverseJoinColumns = { @JoinColumn(name = "aId") })
  // Alternatively, if a Foreign Key or column is used
  // instead of DRY-JOIN table.
  // @OneToMany(fetch = FetchType.LAZY, mappedBy="a_fk")
  private Set<A> manyA;
}

DROP TABLE IF EXISTS A;
CREATE TABLE A (
  id INT(11) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (id)
);


DROP TABLE IF EXISTS B;
CREATE TABLE B (
  id INT(11) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (id)
);

-- Make sure to verify the field constraints when 
----using a DRY-JOIN table that's also managed by Hibernate
DROP TABLE IF EXISTS B_A;
CREATE TABLE B_A (
  id INT(11),
  bId INT(11),
  aId INT(11)
);

Many to One

Example: Owner and Pet (there are many Pets that can be owned by an Owner). And, below many B are mapped to a single A.

Assuming an FK exists:

@Entity
@Table(name = "A")
public class A {

  @Id
  @Column(name = "A_id")
  @GeneratedValue
  private int id;
}

@Entity
@Table(name = "B")
public class B {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  @ManyToOne
  @JoinColumn(name="A_id",foreignKey=@ForeignKey(name="A_id"))
  // Alternatively, if no Foreign Key Constraints exist.
  // @JoinColumn(name="A_id")
  private A a;
}

DROP TABLE IF EXISTS A;
CREATE TABLE A (
  A_id INT(11) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (A_id)
);


DROP TABLE IF EXISTS B;
CREATE TABLE B (
  id INT(11) NOT NULL AUTO_INCREMENT,
  A_id INT(11),
  PRIMARY KEY (id)
);

Many to Many

Example: Books and Authors.

Assuming an FK exists:

@Entity
@Table(name = "A")
public class A {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  @ManyToMany
  @JoinTable(name= "A_B", 
    joinColumns = @JoinColumn(name = "aId"),
    inverseJoinColumns = @JoinColumn(name = "bId"))
  private Set<B> manyB;
}

@Entity
@Table(name = "B")
public class B {

  @Id
  @Column(name = "id")
  @GeneratedValue
  private int id;

  @ManyToMany(mappedBy="manyB")
  private Set<A> manyA;
}

DROP TABLE IF EXISTS A;
CREATE TABLE A (
  id INT(11) NOT NULL AUTO_INCREMENT
  PRIMARY KEY (id)
);


DROP TABLE IF EXISTS B;
CREATE TABLE B (
  id INT(11) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (id)
);

-- Make sure to verify the field constraints when 
----using a DRY-JOIN table that's also managed by Hibernate
DROP TABLE IF EXISTS A_B;
CREATE TABLE A_B (
  id INT(11),
  aId INT(11),
  bId INT(11)
);

Jackson JSON Serialization

To avoid infinite recursion:

Use @JsonIgnore to on side side of the infinite recursion.
Use a custom serializer.
Use @JsonView(Views.Internal.class).

Refer to: https://www.baeldung.com/jackson-bidirectional-relationships-and-infinite-recursion

Best Practices

Use fetch = FetchType.LAZY when configuring Many-to-Many, Many-to-One, and One-to-Many relationships.
Prefer Foreign Key Constraints over a column that refers to a Primary Key or other unique identifier.
Use some annotation like @JsonIgnore to prevent infinite JSON serialization when using Jackson.

Resources and Links

Code samples:

Spring: Tests

Spring Integration Tests are foremost characterized by the injection of the WebApplicationContext:
```
 @Autowired
 private WebApplicationContext wac;
```
- WebApplicationContext spins up a test Application Context so Services, Controllers, etc. are called as they are (not in isolation from each other).
- Spring Mocks will also be used in Integration Tests.
Spring Unit Tests don't spin up a test WebApplicationContext and rely heavily on Spring Mocks to achieve test isolation.

Example Integration Tests

import com.thoughtscript.springunit.config.AppConfig;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import org.springframework.web.context.WebApplicationContext;

import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { AppConfig.class })
@WebAppConfiguration
public class ExampleControllerIntegrationTest {

    @Autowired
    private WebApplicationContext wac;

    private MockMvc mockMvc;

    @Before
    public void preTest() throws Exception {
        mockMvc = MockMvcBuilders.webAppContextSetup(this.wac).build();
    }

    @Test
    public void test() {
        try {
            // Actually calls the endpoint
            mockMvc.perform(get("/test/fetch"))
                    .andDo(print())
                    .andExpect(status().isOk())
                    .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
                    .andExpect(jsonPath("$.text").value("Hello You!"));

        } catch (Exception e) {
            System.out.println("Exception: " + e);
        }
    }

    @After
    public void postTest() {
        mockMvc = null;
    }
}

Example Unit Tests

import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

public class ExampleControllerUnitTest {

    private MockMvc mockMvc;

    @Mock
    private ExampleService exampleService;

    @InjectMocks
    private ExampleController exampleController;


    @Before
    public void preTest() {
        MockitoAnnotations.initMocks(this);
        mockMvc = MockMvcBuilders.standaloneSetup(exampleController).build();
    }

    @Test
    public void test() {
        try {
            // Mocks the endpoint and service
            when(exampleService.helloText()).thenReturn("Hello You!");

            mockMvc.perform(get("/test/fetch"))
                    .andDo(print())
                    .andExpect(status().isOk())
                    .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
                    .andExpect(jsonPath("$.text").value("Hello You!"));

            verify(exampleService, times(1)).helloText();
            verifyNoMoreInteractions(exampleService);

        } catch (Exception e) {
            System.out.println("Exception: " + e);
        }
    }

    @After
    public void postTest() {
        mockMvc = null;
    }
}

Spring: Jupiter Tests

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <modelVersion>4.0.0</modelVersion>
    <groupId>io.thoughtscript.example</groupId>
    <artifactId>spring-cucumber</artifactId>
    <packaging>jar</packaging>
    <version>1.0.0</version>
    <name>spring-cucumber</name>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.1</version>
    </parent>

    <properties>
        <!-- Java Version Related -->
        <java.version>18</java.version>
        <maven.compiler.target>${java.version}</maven.compiler.target>
        <maven.compiler.source>${java.version}</maven.compiler.source>

        <!-- Dependency Versions -->
        <spring.boot.version>3.2.1</spring.boot.version>
        <lombok.version>1.18.30</lombok.version>
        <cucumber.version>7.15.0</cucumber.version>
        <junit-jupiter.version>5.10.1</junit-jupiter.version>
        <junit-platform-suite.version>1.10.1</junit-platform-suite.version>
    </properties>

    <dependencies>

        <!-- Spring Starter Dependencies -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>${spring.boot.version}</version>
        </dependency>

        <!-- Lombok Logging Dependencies -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>${lombok.version}</version>
            <scope>provided</scope>
        </dependency>

        <!-- JUnit 5 (Jupiter) Dependencies -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>${junit-jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <version>${junit-platform-suite.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Cucumber Dependencies -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-spring</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit-platform-engine</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Spring WebMvc Testing -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!-- Inject Mocks Testing -->
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-junit-jupiter</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>

            <!-- Required for mvn spring-boot:run command -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <classpathPrefix>lib/</classpathPrefix>
                            <mainClass>io.thoughtscript.example</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Cucumber Acceptance Testing

package io.thoughtscript.example.acceptance;

import io.cucumber.spring.CucumberContextConfiguration;
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.IncludeEngines;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;
import org.springframework.boot.test.context.SpringBootTest;

import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;

// Looks like this assumes the root dir test/resources/...
@SelectClasspathResource("features")
// This the test package containing the actual Java Step Definition Classes
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "io.thoughtscript.example.acceptance")

// These are required for Cucumber to get picked up by Jupiter during maven-sure-fire.
@Suite
@IncludeEngines("cucumber")

// These are required by Spring
@CucumberContextConfiguration
@SpringBootTest()
public class CucumberAcceptanceTests {
}

package io.thoughtscript.example.acceptance;

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import lombok.extern.slf4j.Slf4j;

import static org.junit.jupiter.api.Assertions.assertEquals;

//These all have to be public visibility
@Slf4j
public class StepDefinitions {

    private int actual;

    //These all have to be public visibility
    @Given("some number crunching")
    public void setup() {
        log.info("prepping...");
    }

    @When("I multiply {int} and {int}")
    public void multiply(Integer x, Integer y) {
        log.debug("Multiplying {} and {}", x, y);
        actual = x * y;
    }

    @When("I add {int} {int} and {int}")
    public void tripleAddition(Integer x, Integer y, Integer z) {
        log.debug("Adding {} {} and {}", x, y, z);
        actual = x + y + z;
    }

    @Then("the result is {int}")
    public void the_result_is(Integer expected) {
        log.info("Result: {} (expected {})", actual, expected);
        assertEquals(expected, actual);
    }
}

Feature: Cucumber Spring Example

  Background: A Basic Example
    Given some number crunching

  Scenario: Multiplication
    When I multiply 4 and 5
    Then the result is 20

  Scenario: Triple Addition
    When I add 1 2 and 3
    Then the result is 6

Refer to: https://github.com/Thoughtscript/spring_cucumber/tree/main/src/test/java/io/thoughtscript/example/acceptance

Controller Tests

package io.thoughtscript.example.controllers;

import io.thoughtscript.example.services.ExampleService;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;

import static org.junit.jupiter.api.Assertions.assertEquals;

@ExtendWith(SpringExtension.class)
@AutoConfigureMockMvc
@SpringBootTest()
class ExampleRestControllerIntegrationTest {

    private final String testString = "OK";

    @Autowired
    ExampleService exampleService;

    @Test
    void testA() {
        assertEquals(testString, exampleService.example());
    }
}

package io.thoughtscript.example.controllers;

import io.thoughtscript.example.services.ExampleService;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;

import static org.junit.jupiter.api.Assertions.assertEquals;

@ExtendWith(SpringExtension.class)
@AutoConfigureMockMvc
@SpringBootTest()
class ExampleRestControllerIntegrationTest {

    private final String testString = "OK";

    @Autowired
    ExampleService exampleService;

    @Test
    void testA() {
        assertEquals(testString, exampleService.example());
    }
}

package io.thoughtscript.example.controllers;

import io.thoughtscript.example.services.ExampleService;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@WebMvcTest(ExampleRestController.class)
class ExampleRestControllerWebMvcTest {

    //This has to be present and will be injected automatically into the ExampleRestController.
    @MockBean
    ExampleService exampleService;

    @Autowired
    private MockMvc mvc;

    @Test
    void testA() throws Exception {
        mvc.perform(MockMvcRequestBuilders
                        .get("/api/example")
                        .accept(MediaType.APPLICATION_JSON))
                .andDo(print())
                .andExpect(status().isOk());
    }

}

Refer to: https://github.com/Thoughtscript/spring_cucumber/tree/main/src/test/java/io/thoughtscript/example/controllers

Basic Tests

package io.thoughtscript.example.helpers;

import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

@Slf4j
class StaticHelpersTest {

    private final String EXPECTED = "invoked";

    @BeforeAll
    // This has to be a static method
    static void init() {
        log.info("JUnit 5 Jupiter tests initializing...");
    }

    @BeforeEach
    void eachInit() {
        log.info("Running before each time...");
    }

    @Test
    // These cannot be private visibility apparently
    void testA() {
            assertEquals(EXPECTED, StaticHelpers.invoke());
    }

    @Test
    void testB() {
        assertNotNull(StaticHelpers.invoke());
    }

    @Test
    void testC() {
        assertEquals(EXPECTED.length(), StaticHelpers.invoke().length());
        assertNotEquals("incorrectString", StaticHelpers.invoke());
    }

    @AfterEach
    void eachAfter() {
        log.info("Running after each time...");
    }


    @AfterAll
    // This has to be a static method
    static void shutdown() {
        log.info("JUnit 5 Jupiter tests completed...");
    }
}

Refer to: https://github.com/Thoughtscript/spring_cucumber/tree/main/src/test/java/io/thoughtscript/example/helpers

Service Tests

package io.thoughtscript.example.services;

import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import static org.junit.jupiter.api.Assertions.*;

@Slf4j
@SpringBootTest
/*
 Apparently auto-wiring into a "basic" Jupiter test
 also requires this annotation now.
*/
public class ExampleServiceWithAutoWiringTest {
    private final String EXPECTED = "OK";

    @Autowired
    ExampleService testService;

    @Test
    void testA() {
        assertEquals(EXPECTED, testService.example());
    }
}

package io.thoughtscript.example.services;

import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

@Slf4j
class ExampleServiceWithoutAutoWiringTest {

    private final String EXPECTED = "OK";

    private ExampleService testService = new ExampleService();

    @Test
    void testA() {
        assertEquals(EXPECTED, testService.example());
    }
}

Refer to: https://github.com/Thoughtscript/spring_cucumber/tree/main/src/test/java/io/thoughtscript/example/services

Resources and Links

https://github.com/Thoughtscript/spring_cucumber/

Spring: Serverless

Resources and Links

Spring: WebFlux

The Reactive paradigm attempts to address shortcomings with blocking, highly-concurrent, systems at scale.

Reactive programming introduces Back-Pressure as the gradual degradation in performance as throughput increases throughout a web service and as information moves through dependencies and internal resources (they use a "water pipe" metaphor - it can get clogged, as water throughput increases pressure increases on the whole system, etc.). Prior paradigms don't handle Back-Pressure very efficiently.

Reactive Programming Principles

A preference for Functional Programming.
Asynchronous Programming from the beginning, not as an afterthought.
Message driven - in line with other coterminous attempts to address the concerns above (Actor-Based systems like Akka, Event Stream, and Messaging systems like Kafka, etc.).
Elasticity - resources are efficiently allocated based on Back-Pressure to reduce performance degradation.

WebFlux Features

Functional Routers - API endpoints that are implemented using Functional Programming.
Streams API oriented.
Mono and Flux as first-class citizens. Promise-like entities available right out of the box.
Backing Reactor daemons to provide Multi-Threaded event loops.

Better performance in highly concurrent use cases.

Prohibitions

One cannot use .block() within any reactive context. Use .subscribe() instead (it's non-blocking but will return a value as an observer).

MongoDB DocumentReferences

Mongo DBRefs aren't supported in Reactive Spring Mongo DB. (One can combine the results of multiple Reactive Streams but that's tedious and unreadable.)
So, use the standard Spring Mongo DB dependencies for managed nesting using the @DBRef annotation.

Refer to: https://docs.spring.io/spring-data/mongodb/docs/3.3.0-M2/reference/html/#mapping-usage.document-references and https://github.com/spring-projects/spring-data-mongodb/issues/3808

Resources and Links

Code samples:

https://github.com/Thoughtscript/spring_2023

Spring: Threads

Spring Threads

Spring Threading manages Threads for the entire web application.
Spring Threads are utilized in processing the actual Business Logic.
Spring Threads use TaskExecutor and TaskScheduler which are implementations of and wrappers for the underlying native Java Executor, ThreadPoolExecutor, etc. (Spring Executors can be used in lieu of the native Java Executors above.)
Spring Executors also execute implementations of Runnable.

Enable Asynchronous task execution via:

@Configuration
@EnableAsync
@EnableScheduling
public class AppConfig {
}

Set the Spring Task Execution Pool defaults:

spring:
  task:
    execution:
      pool:
        core-size: 2
        max-size: 4

Refer to: https://docs.spring.io/spring-framework/docs/4.2.x/spring-framework-reference/html/scheduling.html

Tomcat Threads

Tomcat Threading manages inbound and outgoing web requests, connections, etc. Tomcat Threads are allocated at the "periphery" of the application - so-called Server Thread Pools that continue to be pooled/allocated even when an application itself is terminated.

Remember that many .war files might live within the same Tomcat deployment - Tomcat Threads are shared by all such deployed applications with the web container.

The configuration below (application.yml) specifies the minimum and maximum number of Threads extant in the Tomcat Thread Pool:

server:
  port: 8080
  tomcat:
    max-connections: 200
    # Tomcat thread pool
    threads:
      min: 2
      max: 4

Refer to: https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html

Resources and Links

Code samples:

https://github.com/Thoughtscript/spring_2023/blob/main/_spring/src/main/java/io/thoughtscript/example/controllers/PasswordlessRestController.java

Spring: Asynchronous Programming

Java Spring accomplishes Asynchronous programming in three primary ways:

CompletableFutures and Futures
Executors, Threading, and the @Async keyword
Asynchronous Messaging

@EnableAsync

Enables Spring support for Asynchronous programming:

@Configuration
@EnableAsync
public class AppConfig implements AsyncConfigurer {

    @Override
    public Executor getAsyncExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(7);
        executor.setMaxPoolSize(42);
        executor.setQueueCapacity(11);
        executor.setThreadNamePrefix("MyExecutor-");
        executor.initialize();
        return executor;
    }

    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new MyAsyncUncaughtExceptionHandler();
    }
 }

@Async

After the above is configured, one can use the @Async keyword to automatically wrap a Bean method with an Executor:

@Async
public void asyncMethodWithVoidReturnType() {
    System.out.println("Execute method asynchronously. " + Thread.currentThread().getName());
}

@Async
public Future<String> asyncMethodWithReturnType() {
    System.out.println("Execute method asynchronously - " + Thread.currentThread().getName());
    try {
        Thread.sleep(5000);
        return new AsyncResult<String>("hello world !!!!");
    } catch (InterruptedException e) {
        //
    }

    return null;
}

The Bean method should be:

Public visibility so that Spring can proxy (an interface for) the Asynchronous method. (Consequently, the method can't be invoked within the same Class since doing so would bypass the proxy.)
Have avoid, Future, or CompletableFuture return type.

Resources and Links

Spring: Techniques

Spring Data Mongo

Remember that for two connections, one should use distinct Mongo configuration techniques like so:

package io.thoughtscript.example.configurations;

import com.mongodb.ConnectionString;
import io.thoughtscript.example.Constants;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import org.springframework.data.mongodb.MongoDatabaseFactory;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.SimpleMongoClientDatabaseFactory;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;

@Configuration
@EnableMongoRepositories(basePackages = "io.thoughtscript.example.repositories")
public class MongoConfiguration {

    @Bean
    public MongoDatabaseFactory mongoDatabaseFactory() {
        return new SimpleMongoClientDatabaseFactory(new ConnectionString("mongodb://localhost:27017/" + Constants.MONGO_DB_NAME));
    }

    @Bean
    public MongoTemplate mongoTemplate() throws Exception {
        return new MongoTemplate(mongoDatabaseFactory());
    }
}

package io.thoughtscript.example.configurations;

import com.mongodb.reactivestreams.client.MongoClient;
import com.mongodb.reactivestreams.client.MongoClients;
import io.thoughtscript.example.Constants;
import io.thoughtscript.example.repositories.LanguageMongoReactiveRepository;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.config.AbstractReactiveMongoConfiguration;
import org.springframework.data.mongodb.core.ReactiveMongoTemplate;
import org.springframework.data.mongodb.repository.config.EnableReactiveMongoRepositories;

@Configuration
@EnableReactiveMongoRepositories(basePackageClasses = {LanguageMongoReactiveRepository.class})
@ComponentScan(basePackages = "io.thoughtscript.example.repositories")
class ReactiveMongoConfiguration extends AbstractReactiveMongoConfiguration {

  @Value("${spring.data.mongodb.host}")
  private String host;
  @Value("${spring.data.mongodb.port}")
  private Integer port;

  @Override
  protected String getDatabaseName() {
    return Constants.MONGO_DB_NAME;
  }

  @Override
  public MongoClient reactiveMongoClient() {
    return MongoClients.create();
  }

  @Bean
  public ReactiveMongoTemplate reactiveMongoTemplate() {
    return new ReactiveMongoTemplate(reactiveMongoClient(), getDatabaseName());
  }

}

Reason: two configuration Classess that have overlapping Beans (say AbstractMongoClientConfiguration and AbstractReactiveMongoConfiguration which both initialize MappingMongoConverter mappingMongoConverter) will fail to initialize correctly in the Application Context.

Refer to: https://www.baeldung.com/spring-data-mongodb-tutorial

Resources and Links

https://www.baeldung.com/spring-data-mongodb-tutorial

Go: General Concepts

Go is a Strongly Typed, Compiled, programming language that supports essential Object Oriented concepts without imposing a rigid, heavy-weight, OOD hierarchy.

The Golang Empty Interface can serve as a kind of Generic:

// GoLang generic
// The empty interface specifies that any type may be passed
func golangGeneric(val interface{}) {
  fmt.Println(val)
}

Export from files and Modules using capitalized Variable Names.

Use make() for Arrays and Maps:

nums := [][]int{{1,1},{0,0},{2,1},{100,200},{15,244}}

var animals = make(map[string]string)

Go has no Ternary Operator.
Variables can be declared in two ways:
- var:
  - Can be used in Global Scope (at the top of a Module/Package).
  - Optional declared Type.
```
var a = 900
```
- :=
  - Short Variable Declaration
  - Can only be declared in some local Scope.
```
b := 100
```

Go: Object Oriented Design

Go uses Structs instead of Classes.

They are most akin to a kind of mutable Java Record or JavaScript Object IMO.

They lack internal Functions ("true Methods") and Constructors like in many other languages.

Inheritance

Go does not have pure Inheritance.

Polymorphism

Polymorphism is achieved through flexible Structs where an attribute plays the role of a Type.

Receiver Functions

type Animal struct {
    animalName string
    foodEaten string
    locomotionMethod string
    spokenSound string
}

func (a Animal) Eat() {
    fmt.Println(a.foodEaten + "\n")
}

func (a Animal) Move() {
    fmt.Println(a.locomotionMethod + "\n")
}

func (a Animal) Speak() {
    fmt.Println(a.spokenSound + "\n")
}

Methods (Functions that live within a Class or Instance) are implemented by Receiver Function pattern.

Visibility

Encapsulation is obtained and Access controlled through several techniques and in-built properties of the language:

Export from files and Modules using capitalized Variable Names. (Non-capitalized Variable Names will not be exported - are functionally private.)
Fields on Structs can be read and altered using familiar dot-notation: p.X + p.Y + p.Z.

Examples

type Animal struct {
    animalName string
    foodEaten string
    locomotionMethod string
    spokenSound string
}

cow := Animal{"cow", "grass", "walk", "moo"}
bird := Animal{"bird", "worms", "fly", "peep"}
snake := Animal{"snake", "mice", "slither" ,"hsss"}

// Note this is a weakly OOP language
// Note the lack of inheritance and constructors

type Point struct {
    X    float32
    Y    float32
    Z    float32
}

// Pseudo constructor/factory
// Note capitalized name indicates "to export"
func NewPoint(X float32, Y float32, Z float32) *Point{

    // Partially initialize object
    p := Point{X:X, Y: Y}
    p.Z = Z

    // Return pointer
    return &p
}

// Pseudo-class methods
// Call by value
// Copy of value made

func AddCoordinates(p Point) float32 {
    return p.X + p.Y + p.Z
}

// Receiver type function
// This is also how visibility is controlled:
// By exporting receiver methods but limiting exporting of structs (by using lower-case names)

func (p Point) AdditionReceiver () float32 {
    return p.X + p.Y + p.Z
}

Resources and Links

Code samples:

https://github.com/Thoughtscript/go_refresh/blob/master/courses/9-structsreceivers/main.go

Go: Asynchronous Programming

Go uses WaitGroups, go routines, and the defer keyowrd to make both Multi-Threaded and Asynchronous Programming possible.

func dine(wg *sync.WaitGroup, philosophers []*Philosopher) {
    defer wg.Done()
    //...

wg := new(sync.WaitGroup)

wg.Add(1)
go dine(wg, philosophers)
wg.Wait()

Channels

The output of a go routine can be sent to a Go Channel:

func sortAsync(wg *sync.WaitGroup, arr []int, c chan []int) {
    defer wg.Done()
    fmt.Printf("Go begin: %v \n", arr)
    for i := 0; i < len(arr) - 1; {
        if arr[i] > arr[i+1] {
            orig := arr[i]
            arr[i] = arr[i+1]
            arr[i+1] = orig
            i = 0
        } else {
            i++
        }
    }
    c <- arr
    fmt.Printf("Go end: %v \n", arr)
}

// Create necessary resources
wg := new(sync.WaitGroup)
c := make(chan []int)
arrs := make([][]int, 4)

//...

for i := 0; i < len(arrs); i++ {
    wg.Add(1)
    fmt.Printf("Sorting: %v \n", arrs[i])
    go sortAsync(wg, arrs[i], c)
}

sOne := <- c

Resources and Links

Code samples:

Python: General Concepts

Identation matters
- Is enforced and will complain loudly if you don't indent correctly!
- Has syntactic and semantic relevance
  - def is one indentation less than code blocks
  - The indentation can alter the meaning of block (whether a code terminates or not for instance)
Dynamically Typed
Python is both Interpreted and Compiled
- It's Interpreted Just in Time
  - So, it's Partially Interpreted
- However, it's also Partially Compiled (.pyc files)
Object Oriented
- Supports Multiple Inheritance which Java does not.
None is the relevant Null-ish Type and Value
- None is Unique (Singleton)
except is the relevant catch keyword.
Comparisons:
- == compares sameness of Values
- is compares by (strict) Object identity (by same hash of the Memory Address)

Resources and Links

https://docs.python.org/3/library/functions.html#id

Code samples:

Python: Data Structures

Numbers

No max int. This makes Python attractive for handling very big numbers.

# https://projecteuler.net/problem=36

import math

if __name__ == "__main__":

    try:

        def is_palindrome(num_str):
            LEN = len(num_str)
            HALF = int(math.floor(len(num_str) / 2))

            for x in range(0, HALF, 1):
                if num_str[x] == num_str[LEN - 1 - x]:
                   continue
                else:
                    #print(num_str + " is not a palindrome")
                    return False

            #print(num_str + " is a palindrome")
            return True

        def to_binary(num):
            return format(num, 'b')

        # print(to_binary(585))

        def solve():

            result = []

            for x in range(0, 1000000, 1):
                num_str = str(x)
                A = is_palindrome(num_str)
                binary = to_binary(x)
                B = is_palindrome(binary)

                if A and B:
                    print("Double-base palindrome found: " + num_str)
                    result.append(x)

            print(result)

            sum = 0

            for x in range(0, len(result), 1):
                sum = sum + int(result[x])

            print("Sum found: " + str(sum))
            return sum

        solve()

    except Exception as ex:

        print("Exception: " + str(ex))

List

Ordered sequence of items.
Does not need to be singly typed (items can be of varying types).

lst = list()
lst.append("example")

if ("test" not in lst):
    print("not in")

if ("example" in lst):
    print("is in")

Tuple

Immutable
Corresponds to the mathematical concept of an ordered N-Tuples.
Ordered.

# 3-tuple
exampleTuple = (1,2,3)

# Access tuples
print(exampleTuple[2])

# Tuples can't be changed - they are immutable
## With throw error if uncommented

### exampleTuple[1] = 44

# Destructuring
(x, y, z) = (9,10,11)
print(x)
print(y)
print(z)
print((x, y, z))

# Comparison - element by element
print(exampleTuple < (x, y, z))
print(exampleTuple > (1000, "", "Hello"))

String

line = "abcdefghijklmnop..."

# indexOf
startIndex = line.find("0.")

# Length of String
endIndex = len(line)

# Slice
line[startIndex:endIndex]

Sets

Share curly brace reserved symbols {,} with Dictionaries but are element-valued only.
Do not guarantee nor preserve order.
Deduplicated.
Corresponds to the mathematical concept of a Set.

thisset = {"apple", "banana", "cherry", "apple"}

Dictionary

Key-Value data structure.
Share curly brace reserved symbols {,} with Sets but are Key-Value.

exampleDictionary = {
    "field": "example",
    "attribute": "another",
    "numerical": 2
}

print(exampleDictionary)

# access value of dict by key
numericalOne = exampleDictionary.get("numerical")
print(numericalOne)

numericalTwo = exampleDictionary["numerical"]
print(numericalTwo)

# set value of dict
exampleDictionary["numerical"] = 4
print(exampleDictionary["numerical"])

# iterating through the dict
## keys
for x in exampleDictionary:
    print(x)

## values
for x in exampleDictionary:
    print(exampleDictionary[x])

for x in exampleDictionary.values():
    print(x)

## keys and values by destructuring
for x, y in exampleDictionary.items():
    print(x, y)

Boolean

Capitalized.

False
True

Resources and Links

Code samples:

Python: Comprehensions

Comperehensions provide syntactic sugar for initializing iterables. They use the following succinct Generator Expression syntax:

Element Comprehensions - x for x in range(0, 9)
Key-Value Comprehensions - x : x + 1 for x in range(0, 9)
- Left of the : specifies the Key and the right determines the resolved Value.

Generator Expressions

Very succinct in terms of written code and memory use. (Consider the extremely verbose initialization of Java nested Arrays with non-default values or static block initializations!)

print([y+y for y in range(0, 10)])
# [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

Consult: https://peps.python.org/pep-0289/

Tuple

Tuple Comprehension isn't supported since Tuples already use Generators (Generator Expressions to be more precise) under the hood.

Review: https://stackoverflow.com/questions/16940293/why-is-there-no-tuple-comprehension-in-python

List

example_list = [x for x in range(0, 9)]
print(example_list) # [0, 1, 2, 3, 4, 5, 6, 7, 8]

Dict

example_dict = { x : x + 1 for x in range(0, 9) }
print(example_dict) # {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9}

Resources and Links

Python: Object Oriented Design

Classes, Inheritance, and Multiple Inheritance

class Dog:
    breed = "grayhound"
    name = "fido"

class Cat:

    # constructor
    def __init__(self, name, breed):
        self.name = name
        self.breed = breed

    # instance method
    def meow(self, meow):
        ## string interpolation
        return "{} says meow {}".format(self.name, meow)

# executable code
cat = Cat("Sonata", "black cat")
print(cat.meow("meow")) # Sonata says meow meow

class RoboCat(Cat):

    # constructor with super
    def __init__(self, name, breed, metal):
        super().__init__(name, breed)
        self.metal = metal

# Multiple Inheritance
class CatDog(Cat, Dog):

    # constructor with super
    def __init__(self, name, breed):
        super().__init__(name, breed)

# executable code
catdog = CatDog("CatDog", "cat dog")
print(catdog.meow("meow")) # CatDog says meow meow

Resources and Links

Code samples:

https://github.com/Thoughtscript/python-refresh/blob/master/examples/4%20-%20dependency/dependency.py

Python: Modules and Packages

Python scripts and files can be organized or grouped into Modules and Packages.

By default, Modules can be imported (using the import keyword) using a Namespace corresponding to the named directory structure.

A Package can also be defined with a customized Namespace (the Namespace itself for instance) and is typified by the presence of __init__.py files at the root level of the Package.

Importing and Dependency Injection

Given:

+- /example
|   +- /dependencies
|       +- a.py
+- dependency.py
+- main.py

/dependencies/a.py:

class B:
    num = 0

dependency.py:

class Dog:
    breed = "grayhound"
    name = "fido"

# multiple classes in one script
class Cat:

    # constructor
    def __init__(self, name, breed):
        self.name = name
        self.breed = breed

    # instance method
    def meow(self, meow):
        ## string interpolation
        return "{} says meow {}".format(self.name, meow)

# executable code
cat = Cat("Sonata", "black cat")
print(cat.meow("meow"))

class RoboCat(Cat):

    # constructor with super
    def __init__(self, name, breed, metal):
        super().__init__(name, breed)
        self.metal = metal

main.py:

import dependency
import dependencies.a as A

if __name__ == '__main__':

    try:
        # dependency injection example
        robo = dependency.RoboCat("cat one", "Egyptian", "chrome")
        print(robo.meow("10101010011"))

        # a second dependency injection example
        B = A.B
        print("A.B " + str(B.num))

    except Exception as ex:

        print('Exception! ' + str(ex))

Review: https://github.com/Thoughtscript/python-refresh/tree/master/examples/4%20-%20dependency

Monkey Patching

Replacing a predefined Function or Method with another one (dynamically):

# Given the above imported classes...

def monkey_patch_function(self):
    print("I'm a monkey now!")

dependency.Cat.meow = monkey_patch_function

monkeycat = Cat("Monkey Cat", "money cat")
monkeycat.meow() # I'm a monkey now!

Packages

Consult: https://github.com/Thoughtscript/python_api/blob/main/server/init/__init__.py

Resources and Links

https://stackoverflow.com/questions/448271/what-is-init-py-for

Code samples:

Python: Error Handling

None

None is the relevant nil, null, and undefined keyword, concept, and Object.

None is Singleton - it is uniquely created and one copy exists.

Use the is keyword to check for None.

if x is None:
    # ...

Errors and Exceptions

In Java, an Error specifies an application-terminating event whereas an Exception is something that deviates from some expectation and ought to be handled in most circumstances.

Python Errors refer syntax-specific concerns. Whereas an Exception is anything thrown (or "raised" via the raise keyword).

try:
    raise NameError('HiThere')

except NameError:
    print('An exception flew by!')

    # rethrow
    raise

https://docs.python.org/3/tutorial/errors.html#raising-exceptions

Exception Handling

Three primary keywords (try, except, and finally) are used to handle a raised Exception:

if __name__ == '__main__':
    try:
        # ...

    except Exception as ex:
        # Executes when error is thrown
        # ...

    else:
        # else can be supplied here
        # even with a subsequent finally clause

    finally:
        # Always executes regardless of thrown error
        # ...

Exception Groupings

Python supports many Exceptions being raised simultaneously without the need for more complex abstraction:

def f():
    # Define a List of Exceptions to pass
    excs = [OSError('error 1'), SystemError('error 2')]
    # raise and throw them all with an ExceptionGroup
    raise ExceptionGroup('there were problems', excs)

# ...

try:
    f()

# Use top-level Exception to catch them in an except clause
except Exception as e:
    print(f'caught {type(e)}: e')

https://docs.python.org/3/tutorial/errors.html#raising-and-handling-multiple-unrelated-exceptions

Resources and Links

Code samples:

Python: Truth Values

Falsy Values

None

False

# Any zero number:
0, 0L, 0.0, 0j

# Any empty sequence:
'', (), []

# Any empty mapping:
{}

Truthy Values

All values that aren't Falsy are considered True.

References and Links

https://docs.python.org/2.4/lib/truth.html

Python: Pass By Assignment

Pass By Assignment

Other monnikers: Call by Object Reference or Call by Assignment.

Python uses a mix of approaches:

Pass By Value - passes the Value and not the Object Reference as exhibited below:

 def call_by_value(val):
     val *= 2
     print("within call_by_value: " + str(val))

 num = 1
 print("before: " + str(num))
 call_by_value(num)
 print("after: " + str(num))

 before: 1
 within call_by_value: 2
 after: 1

num is not modified in the outer scope.

Pass By Reference - when the entire Object is passed per the following:

 def call_by_ref(exm_mp):
     exm_mp["a"] = "b"
     print("within call_by_ref: " + str(exm_mp))

 exm_mp = {}
 print("before: " + str(exm_mp))
 call_by_ref(exm_mp)
 print("after: " + str(exm_mp))

 before: {}
 within call_by_ref: {'a': 'b'}
 after: {'a': 'b'}

exm_mp is modified in the outer scope and retains changes made within the Function.

Pass By Assignment - a species of Pass By Reference that's specific to Python.

Values can be associated with multiple References (similar to the way Java's String Pool allows the same String Value to be used for multiple String Variables).

 from sys import getrefcount

 print("--- Before  assignment ---")
 print(f"References to 1: {getrefcount(1)}")
 print(f"References to 2: {getrefcount(2)}")
 x = 1
 print("--- After   assignment ---")
 print(f"References to 1: {getrefcount(1)}")
 print(f"References to 2: {getrefcount(2)}")
 x = 2
 print("--- After reassignment ---")
 print(f"References to 1: {getrefcount(1)}")
 print(f"References to 2: {getrefcount(2)}")
 print("--- Addresses in Memory ---")
 print(id(x))
 print(id(1))
 print(id(2))
 print("--- Comparisons ---")
 print(id(x) == id(1))
 print(id(x) == id(2))
 print(x is 1)
 print(x is 2)
 print(x == 1)
 print(x == 2)

Notice that the count changes based on how x is set. Also, the high numer of References to 1 and 2 even before x References either.

 main.py:21: SyntaxWarning: "is" with a literal. Did you mean "=="?
   print(x is 1)
 main.py:22: SyntaxWarning: "is" with a literal. Did you mean "=="?
   print(x is 2)
 --- Before  assignment ---
 References to 1: 129
 References to 2: 111
 --- After   assignment ---
 References to 1: 130
 References to 2: 111
 --- After reassignment ---
 References to 1: 129
 References to 2: 112
 --- Addresses in Memory ---
 140174349960512
 140174349960480
 140174349960512
 --- Comparisons ---
 False
 True
 False # See Warning above
 True # See Warning above
 False
 True

Comparing Objects

Can use id() to find the unique identifier for an Object (the Address of the Object in memory):

num = 1
print(id(num)) # 132506274753480

(Object Reference is also used within is comparison checks.)

Consult: https://www.w3schools.com/python/ref_func_id.asp

Documentation: https://docs.python.org/3/library/functions.html#id

References and Links

Python: Asynchronous Libraries

asyncio

Uses the familiar keywords and syntax of async, await in JavaScript:

import asyncio

async def count(run_var):
    print(run_var + "One")
    await asyncio.sleep(1)
    print(run_var + "Two")

# Must run and call async functions with await
async def main_method():
    await asyncio.gather(count("a"), count("b"), count("c"))

if __name__ == "__main__":
    import time
    s = time.perf_counter()
    # Use asyncio runner
    # https://docs.python.org/3/library/asyncio-runner.html
    asyncio.run(main_method())
    elapsed = time.perf_counter() - s
    print(f"{__file__} executed in {elapsed:0.2f} seconds.")

aOne
bOne
cOne
aTwo
bTwo
cTwo

Resources an Links

Python: Techniques

Commenting

'''
I'm a multiline docstring!

I can be accessed via introspection: __doc___
'''

# I'm a single line comment

Review: https://docs.python.org/3/glossary.html#term-docstring

Float and Floor Division

'''
Float Division 
- Results in a float 
'''
5 / 2 # 2.5

'''
Floor Division 
- Results in an integer rounded down
'''
5 // 2 # 2

// behaves like Integer or Math.round() division in Java / JavaScript.

Consult also: https://docs.python.org/3/library/functions.html#divmod

args and kwargs

Is similar to Java Generic, Go Empty Interface, and Spread Operator ( ...) in JavaScript.

# Any number can be passed - variable length
# Can also just pass a List
def my_func_a(*args_example):
    for arg in args_example:
        print(arg)

# Use
my_func_a('a', 'b', 'c')

# Use in tandem with other required arguments
def my_func_b(arg, *args_example):
    for argv in args_example:
        print(argv)

# Use
my_func_b('arg', 'a', 'b', 'c', 'd')

# Use kwargs to pass in a varying number of arguments
# These are key-value pairs
def my_func_c(**kwargs):
    for key, value in kwargs.items():
        print("Key: {0} Value: {1}".format(key, value))

# Use
my_func_c(a='a', b='b', c='c', d='d')

Consider: https://nodejs.org/api/process.html#processargv process.argv in Node.

Pickling and Unpickling

Akin to Java's transient keyword which uses a customized hashing mechanism to divide a value into files and persist them.

The Pickle Module saves an Object or Value as a String within a File.

Unpickling involves deserializing that value back into an in-memory Python Object or Value.

Pass

pass can be supplied in leiu of code block:

def example_a():
    pass

# note the following isn't syntactically valid
def example_b():
    # nothing here

REST API

Using Flask and backing SQL database:

from init import get_app
from flask import request
from domain import scan_examples, get_example, delete_example, create_example, update_example

"""
DB endpoints.
"""

app = get_app()

@app.route("/api/db/example", methods=['POST'])
def postExample():
    name = request.form.get('name', '')
    result = create_example(name)

    return [str(result)]

@app.route("/api/db/examples", methods=['GET'])
def scanExamples():
    results = scan_examples()
    response = []

    for x in results:
        response.append(str(x))

    return response

@app.route("/api/db/example/<id>", methods=['PUT'])
def updateExample(id):
    name = request.form.get('name', '')
    result = update_example(id, name)

    return [str(result)]

@app.route("/api/db/example/<id>", methods=['GET'])
def getExample(id):
    result = get_example(id)

    return [str(result)]

@app.route("/api/db/example/<id>", methods=['DELETE'])
def deleteExample(id):
    result = delete_example(id)

    return [str(result)]

from init import db

"""
Domain.
"""

class Example(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=False)

    def __init__(self, name):
        self.name = name

    # Must be valid JSON serializable string
    def __repr__(self):
        return '{ id: %s, name: %r }' %(self.id, self.name)

"""
Queries.
"""

def scan_examples():
    return Example.query.all()

def get_example(id):
    return Example.query.get(id)

def delete_example(id):
    example = Example.query.get(id)
    db.session.delete(example)
    db.session.commit()
    return example

def update_example(id, name):
    example = Example.query.get(id)
    example.name = name
    db.session.commit()

    result = Example.query.get(id)
    return result

def create_example(name):
    example = Example(name)
    db.session.add(example)
    db.session.commit()

    return example

def prepopulate():
    db.drop_all()

    db.create_all()

    example_a = Example('example_a')
    db.session.add(example_a)

    example_b = Example('example_b')
    db.session.add(example_b)

    example_c = Example('example_c')
    db.session.add(example_c)

    example_d = Example('example_d')
    db.session.add(example_d)

    db.session.commit()

Review: https://github.com/Thoughtscript/python_api/tree/main/server

Resources and Links

Code samples:

https://github.com/Thoughtscript/python_api/tree/main/server

JavaScript: Comparisons

Strict

Checks for exactly the same Value and Type:

a === b

Loose

Checks for the same Value after automatic conversion to a common Type (shared Type Coercion):

a == b

Object.is()

Special comparison for NaN and 0 along with some other important but "edgy" cases.

Refer to: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Comparison_Operators

Type Checking

There are three primary ways to Type Check:

Primitive checking:

typeof 'Hello World' === 'string';

Check whether an Object is an instance of a Class:

var x = new Dog();
x instanceof Dog;

Via constructor:

var x = 'Hello World';
x.constructor === String;

JavaScript: This Keyword

The this keyword allows one to manipulate the current Closure/scope.

Arrow Functions

Note that JavaScript Arrow Functions don't bind this to their Closure/scope (this will refer to the next-nearest available surrounding scope).

const A = function() {

    this.example = "hello, world"

    const B = () => {
        // Refers to the scope of A
        console.log(this.example) 
    }

    B()
}() // "hello, world"

Scope Binding

Consider the following:

var meeple = "Meeple One"

var myObj = {
    meeple: "Meeple Two",
    prop: {
        getMeeple: function() {
            return this.meeple
        },
        meeple: "Meeple Three"
    }
}

console.log(myObj.prop.getMeeple())

var test = myObj.prop.getMeeple

console.log(test())

The output will be:

"Meeple Three"
"Meeple One"

Remember that this gets bound to the Scope of the function called - in the case of test() the Scope is the outer Closure since it's defined in the top-level Scope (of the script file itself).

JavaScript: Falsy Values

Are interpreted as false:

    false  
    0 (zero)  
    "" (empty string)  
    null  
    undefined    
    NaN

All values are truthy unless they are falsy.

JavaScript: Variables

let - specifies a mutable variable declaration, scoped to the immediate local Closure scope.
const - immutable once declared, specifies a Constant.
var - original, general, not immutable, not scoped, Hoisted.

Advantages of Let vs. Var

Since let is scoped to a local context, doing the following:

for (let i = 0; i < els.length; i++) {
    els[i].addEventListener('click', function() {
        console.log(i)
    })
}

will correctly print the right number i. If var is used instead each button will only print els.length - 1.

JavaScript: Prototypes

JavaScript's Object Oriented

Examples

var Human = function(name) {
    this.name = name;
}

Human.prototype.getName = function() {
    return this.name;
}

Note that Prototype names must be written using Pascal Notation.

Object Oriented Design

Define a Prototype:

var Dog = function(bark) {
    this.bark = bark;
};

//Call constructor
var p = new Dog("woof");

// Add a method to Dog.prototype
Dog.prototype.tail = function(){
    console.log("wags");
};

Inheritance:

function Poodle(bark, name) {
    // Call the parent constructor
    Dog.call(this, bark);
    this.name = name;
};

Or explicitly set prototypical inheritance using Object.create():

//Explicitly set prototypical inheritance
//Allows overriding parent methods
Poodle.prototype = Object.create(Dog.prototype);

Test it out:

var d = new Poodle("bark bark", "dooder");
d.tail(); //"wags"

console.log(d instanceof Dog);  // true 
console.log(d instanceof Poodle); // true

JavaScript: Promises

Promises - an Object for handling Asynchronous, non-blocking, operations.
Async/Await - syntactic sugar for returning and handling a Promise

Examples

Must wrap all await keywords within an async method.

const asyncFunc = async (x) => {
    return 2
}

const EX = async () => {
    const V = await asyncFunc(1)
    console.log(V) //2
}

EX()

const assignAsyncVal = () => new Promise((resolve, reject) => {
    asyncFunc(1).then(success => {
        return resolve(success)
    })
})

const V = assignAsyncVal().then(success => {
    console.log(success) //2
})

Using await with the Promise function assignAsyncVal() above:

const EX = async () => {
    const VV = await assignAsyncVal()
    console.log(VV) //2
}

EX()

Many Promises

To resolve multiple Promises, push them into an Array and use Promise.all(), Promise.allSettled(), etc.

const A = new Promise((resolve, reject) => {})
const B = new Promise((resolve, reject) => {})

const PROMISES = []
PROMISES.push(A)
PROMISES.push(B)

// Resolve all promises
PROMISES.all(success => {
    //...
})

Resources and Links

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

JavaScript: Asynchronous Loops

Given the following helper method:

const print = str => {
  const el = document.getElementById("here");
  const txt = document.createTextNode(str);
  el.append(txt);
  const br = document.createElement("br");
  el.append(br);
  console.log(str);
};

Synchronous Loops

A few examples.

const firstExample = () => {
  let i;
  for (i = 0; i < 10; i++) {
    print("1st middle: " + i);
  }
  print("1st end: " + i);
};
firstExample();

const secondExample = () => {
  let i;
  for (i = 0; i < 10; i++) {
    print("2nd middle: " + i);
  }
  print("2nd end: " + i);
  return i;
};
let j = 0;
j = secondExample();
print("J 2nd: " + j);

const fourthExample = () => {
  let y = 0;
  for (let i = 0; i < 10; i++) {
    y++;
    print("4th middle: " + y);
  }
  print("4th end: " + y);
  return y;
};

let v = 0;
v = fourthExample();
print("V 4th: " + v);

1st middle: 0
1st middle: 1
1st middle: 2
1st middle: 3
1st middle: 4
1st middle: 5
1st middle: 6
1st middle: 7
1st middle: 8
1st middle: 9
1st end: 10

2nd middle: 0
2nd middle: 1
2nd middle: 2
2nd middle: 3
2nd middle: 4
2nd middle: 5
2nd middle: 6
2nd middle: 7
2nd middle: 8
2nd middle: 9
2nd end: 10
J 2nd: 10

4th middle: 1
4th middle: 2
4th middle: 3
4th middle: 4
4th middle: 5
4th middle: 6
4th middle: 7
4th middle: 8
4th middle: 9
4th middle: 10
4th end: 10
V 4th: 10

Asynchronous Loops

Given:

const thirdExample = () => {
  return setTimeout(() => {
    let y = 0;
    for (let i = 0; i < 10; i++) {
      y++;
    }
    print("3rd end: " + y);
    return y;
  }, 1000);
};
let z = 0;

z = thirdExample();
print("Z 3rd: " + z);

Intuitively, we think that the output should be something like:

// Incorrect assumption
3rd end: 10
Z 3rd: 10

Given:

const fifthExample = () => {
  let y = 0;
  setTimeout(() => {
    y = 10000;
  }, 4000);
  return y;
};

print("5th: " + fifthExample());

It's natural to think the output will be:

// Incorrect assumption
5th: 10000

The actual chronological output:

Z 3rd: 1

5th: 0

3rd end: 10

To remedy those counter-intuitive outputs wrap the methods with a Promise.

Event Stack

for (let i = 0; i < 3; i++) {
   console.log(1)
   setTimeout(() => console.log(i), 0)
   setTimeout(() => console.log(2), 0)
   setTimeout(() => console.log(3), 100)
   console.log(4)
}

In chronological order:

Perhaps counter-intuitively:

Both console.log(1) and console.log(4) print before the rest (even the setTimeout() calls with 0 delay). This has to do with the way that JavaScript handles Events on the Stack. setTimeout() calls are lower priority Events and are executed last, even with a 0 delay.
One might think that all console.log(3) calls would print prior to each console.log(4). They all occur after the fact. JavaScript essentially ignores any asynchronous blocking behavior within a loop.

Button Event Listeners

Use let instead of var.
Always lookup "fresh" info within the actual event callback rather than supplying some info at initialization since it will likely both be (1) out-of-date and (2) asynchronously invalid.

Refer to the article on JavaScript Variables.

JavaScript: Deep and Shallow Copies, Merging

Deep Copying creates a copy of an object that share no references in memory.

Shallow Copying creates a copy of an object that shares a reference in memory.

Deep Merging allows the deeply nested fields of two objects to be added to, combined, or merged with one of those objects.

From the Reference Documentation

“A deep copy of an object is a copy whose properties do not share the same references (point to the same underlying values) as those of the source object from which the copy was made. As a result, when you change either the source or the copy, you can be assured you're not causing the other object to change too; that is, you won't unintentionally be causing changes to the source or copy that you don't expect. That behavior contrasts with the behavior of a shallow copy, in which changes to either the source or the copy may also cause the other object to change too (because the two objects share the same references).”

Shallow Copying

Consider the following tensor of rank 1, a single dimension array:

const X = [1,2,3,4,5,6,7,8]
const Y = [...X]
X[0] = 1000
console.log(X) // [ 1000, 2, 3, 4, 5, 6, 7, 8 ]
console.log(Y) // [ 1, 2, 3, 4, 5, 6, 7, 8 ]

When the array being duplicated is of rank > 1:

const A = [[1,2,4,5,6,7,8],[1,2,4,5,6,7,8]]
const B = [...A]
A[0][0] = 1000
console.log(A) // [ [ 1000, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]
console.log(B) // [ [ 1000, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]

Deep Copying

The recommended way now is to use:

JSON.parse(JSON.stringify(ingredients_list)) // Object

const A = [[1,2,4,5,6,7,8],[1,2,4,5,6,7,8]] // Tensor of rank 2, matrix, m x n array where n > 1

// By element
const R = []
for (let r = 0; r < A.length; r++) {
    const row = []
    for (let c = 0; c < A[r].length; c++) {
            row.push(A[r][c])
    }
    R.push(row)
}

R[0][0] = 1000
console.log(R) // [ [ 1000, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]
console.log(A) // [ [ 1, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]

const A = [[1,2,4,5,6,7,8],[1,2,4,5,6,7,8]] // Tensor of rank 2, matrix, m x n array where n > 1

// By row spread operator
const R = []

for (let r = 0; r < A.length; r++) {
    // Single dimensioned arrays are deep copied by the spread operator.
    R.push([...A[r]])
}

R[0][0] = 1000
console.log(R) // [ [ 1000, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]
console.log(A) // [ [ 1, 2, 4, 5, 6, 7, 8 ], [ 1, 2, 4, 5, 6, 7, 8 ] ]

Deep Merging

Object.assign({}, obj)

Resources and Links

https://developer.mozilla.org/en-US/docs/Glossary/Deep_copy

JavaScript: General Concepts

JavaScript is an interpreted, often transpiled, Object Oriented programming language.
Prototype-based Object Oriented Design with optional Class syntactic sugar.
Dynamically Typed - JavaScript uses Type Coercion to Dynamically Type values and variables at Run Time.
Technically, "JavaScript" is a loose name for a family of dialects that implement the ECMAScript standard: https://www.ecma-international.org/technical-committees/tc39/ - officially, there is no "JavaScript" but the name persists.
JavaScript Primitive Data Types (Number, String, BigDecimal, Boolean, etc.) Pass by Value. Arrays and Objects Pass by Reference (one cause for the phenomenon of Shallow Copying).

Use Strict

Strict Mode enforces stricter/more secure execution of a Script at Run Time:

Hoisting and assignment to variables that weren't declared result in syntax errors.
this is Autoboxed more stringently.
eval() is handled more securely.
Silent errors that are typically ignored are thrown verbosely.

'use strict'

//...

Refer to: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Strict_mode

JavaScript: Node

Example main method wrapped with try-catch clauses and Proccess exception handlers:

'use strict'

try {
  process.on('warning', warning => { console.error(`Warning encountered: ${warning}`) })
  process.on('unhandledRejection', rej => { console.error(`Unhandled Rejection override: ${rej}`) })
  process.on('uncaughtException', exception => { console.error(`Error encountered: ${exception}`) })
  process.on('exit', msg => { console.log(`Service shutting down: ${msg}`) })

//...

} catch (ex) {
  console.error(`Exception ${ex}!`)
  process.exit()
}

Workers

const {Worker } = require('worker_threads')

const createWorker = (filename, paramsToExecute) => new Promise((resolve, reject) => {

    //Constructor
    //Will pass paramsToExecute to the method executed in filename
    //Must have workerData as attribute
    const W = new Worker(filename, {workerData: paramsToExecute })

    //Listeners in parent thread
    W.on('message', message => {
        console.log(`Worker message received: ${message}!`)
        resolve(message)
    })

    W.on('error', error => {
        console.error(`Worker error encountered: ${error}!`)
        reject(error);
    })

    W.on('exit', exitCode => {
        if (exitCode !== 0) {
            console.error(`Worker stopped with exit code ${exitCode}`)
            reject(exitCode)
        } else {
            console.log(`Worker stopped with exit code ${exitCode}`)
            resolve(exitCode)
        }
    })

    //Send message to worker script
    W.postMessage('I am initialized...')

})

//Wrap this with a Thread Pool and/or Thread count to prevent excessive resourcing
const executeServiceUsingThread = (filename, paramsToExecute) => new Promise((resolve, reject) => {
    createWorker(filename, paramsToExecute).then(success => {
        console.log(`Service completed: ${success}!`)
    }, failure => {
        console.error(`Service completed: ${failure}!`)
    })
})

module.exports = {
    executeServiceUsingThread: executeServiceUsingThread
}

"use strict"

/**
 * Note: the WebWorker importScripts() cannot be used here.
 *
 * These are essentially NodeJS compliant scripts
 */

const { workerData, parentPort } = require('worker_threads')

//console.log test

console.log('Test console log inside Worker One')
console.log(`Getting Worker One workerData: ${workerData}`)

const conversationMappings = {
    "Hello!": "Goodbye!"
}

const exampleDependencyFunction = text => conversationMappings[text]

parentPort.postMessage(`Worker one: ${workerData} - message response: ${exampleDependencyFunction(workerData)}!`)

//...
WT.executeServiceUsingThread('./nodeThread/workerOne/worker_script_one.js', "Hello!")

HTTP

module.exports = {
  createHttpServer: () => {
    const S = require('http').createServer(require('./express').createServer())

    console.log('HTTP initialized!')
    console.log('REST API controller initialized!')

    S.on('clientError', (err, sck) => {
      const e = `HTTP/1.1 400 Bad Request! ${err}`
      console.error(e)
      sck.end(e)
    })

    S.listen(require('../config').SERVER.PORT, () => { console.log(`HTTP server started on port: ${S.address().port}`) })

    return S
  }
}

HTTPS

With Express:

const C = require('../config'), FS = require('node:fs')

module.exports = {
  createHttpsServer: () => {
    const OPTS = {
      key: FS.readFileSync(C.SERVER.SSL.KEY_PATH),
      cert: FS.readFileSync(C.SERVER.SSL.CERT_PATH)
    }

    const S = require('node:https').createServer(OPTS, require('./express').createServer())

    console.log('HTTPS initialized!')

    S.listen(C.SERVER.HTTPS_PORT, () => { console.log(`HTTPS server started on port: ${S.address().port}`) })

    return S
  }
}

const express = require('express'),
  C = require('../config')

module.exports = {
  createServer: () => {
    const app = express()

    app
      .use(require('morgan')('dev'))
      .use(express.json())
      .use(express.urlencoded({ extended: true }))

      .use(require('cookie-parser')())

      .use(require('cors')({
        origin: C.SERVER.CORS,
        optionsSuccessStatus: 200
      }))

      .use('/api', require('./api'))

    return app
  }
}

const express = require('express'),
  privateApi = express.Router(),
  C = require('../config')

privateApi
  .all('*', async (req, res, next) => await require('./auth').LOG_IP_FILTER(req, res, next))

  .all('*', async (req, res, next) => await require('./auth').DECRYPT(req, res, next))

  .all('*', async (req, res, next) => await require('./auth').AUTH_CHECK(req, res, next))

  // https://localhost:8888/api/projections?auth=...
  .get("/projections", async (req, res) => {
    let responseData = await require('./domain/budgetprojections').BUDGET_PROJECTIONS({})
    let stringData = require('./auth').SCRAMBLE(C.AUTH.V(), C.AUTH.BP(), JSON.stringify(responseData))
    return res.send({ status: 200, data: stringData })
  })

  //...

Exports and Imports

module.exports = {
    myExportedFunction: () => {
      //...
    }
}

const R = require('../path/to/my/file.js')

R.myExportedFunction()

Resources and Links

Code samples:

JavaScript: Web Workers

WorkerManager

<html>
<head>
    <meta charset='UTF-8'>
    <title>WebWorker Example</title>
</head>
    <body>
        <h3>WebWorker Request:</h3>
        <h id="request">Hello!</h>
        <h3>WebWorker Responses:</h3>
        <p id="response"></p>
    </body>
    <script type="text/javascript" src="workerManager.js"></script>
    <script>
        var wm = new WorkerManager();
        wm.startWrapperExample(document.getElementById("request").textContent);
    </script>
</html>

Using a WorkerManager wrapper with two example implementations: BlobWorker and ScriptWorker.

//workerManager.js

"use strict";

/** BEGIN WorkerManager Prototype */

var WorkerManager = function () {
};

WorkerManager.prototype.Worker = null;

WorkerManager.prototype.opts = {
    inline: false,
    scriptWorker: {
        script: "scriptWorker.js"
    },
    blobWorker: "self.addEventListener('call', function(e) {" +
    "self.addEventListener('message', function(e) {" +
    "self.postMessage(exampleDependencyFunction(e.data));" +
    "}, false);" +
    "}, false);"
};

WorkerManager.prototype.createBlobWorker = function () {
    var blob = window.URL.createObjectURL(new Blob([WorkerManager.prototype.blobWorker]));
    this.Worker = new Worker(blob);
};

WorkerManager.prototype.createScriptWorker = function () {
    this.Worker = new Worker(this.opts.scriptWorker.script);
};

WorkerManager.prototype.isSupported = function () {
    return window.Worker;
};

WorkerManager.prototype.startWrapperExample = function (t) {
    if (this.isSupported()) {
        if (this.opts.inline) this.createBlobWorker();
        else this.createScriptWorker();
        if (this.Worker != null) {
            this.Worker.postMessage(t);
            this.Worker.addEventListener('message', function (e) {
                document.getElementById('response').textContent = e.data;
            }, false);
        }
    }
};

/** END WorkerManager Prototype */

ScriptWorker

//scriptWorker.js

"use strict";

importScripts('./scriptWorkerDependency.js');

self.addEventListener('message', function (e) {
    self.postMessage(exampleDependencyFunction(e.data));
}, false);

ScriptWorker Dependency

// scriptWorkerDependency.js

"use strict";

var conversationMappings = {
    "Hello!": "Goodbye!"
};

var exampleDependencyFunction = function(text) {
    return conversationMappings[text];
};

JavaScript: Techniques

Initialize Many EventListeners

To initialize an EventListener on a DOM Element:

const el = document.getElementById('todos')

for (let i = 0; i < el.children.length; i++) {
    const c = el.children[i]

    c.addEventListener('click', (e) => {
        console.log(e.target.id)
        e.target.style.display = 'none'
        e.preventDefault
    })
}

Remember that by the time the Element is clicked:

c will lose its value - it's disassociated with the Element by that point.
e.target is still the preferred, official, way to access the Attributes on the Element, the Element itself, etc.
this will refer to the top-level scope and shouldn't be used.

In short, use e and e.target to access the propagated DOM Event and current Element, respectively. (Similar to React's ref.current.)

Fastest Way To Remove An Index

Supposedly the fastest way to remove an index in an array is as follows:

const todos = []

for (let i = 0; i < todos.length; i++) {
    if (todos[i].uuid === uuid) {
        // Remove by index
        todos.splice(i, 1)
        break
    }
}

Vs. deep removal:

const test = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0, "a", 'b', 'c', 'd', 'e', "f", 'g']

const deepRmv = key => {
  const temp = [...test]
  let result = []

  const BEGIN = new Date()
  for (let i = 0; i < temp.length; i++) {
    if (temp[i] === key) continue
    result.push(temp[i])
  }
  const END = new Date()

  console.log(`deepRmv time: ${END-BEGIN} result: ${result}`)

  return result
}

const rmvBySlice = key => {
  const temp = [...test]

  const BEGIN = new Date()
  for (let i = 0; i < temp.length; i++) {
    if (temp[i] === key) {
      temp.splice(i, 1)
      break
    }
  }
  const END = new Date()

  console.log(`rmvBySlice time: ${END-BEGIN} result: ${temp}`)

  return temp
}

deepRmv("d")
rmvBySlice("d")

"deepRmv time: 0 result: 1,2,3,4,5,6,7,8,9,0,a,b,c,e,f,g"
"rmvBySlice time: 0 result: 1,2,3,4,5,6,7,8,9,0,a,b,c,e,f,g"

Probably best to use for loop with splice since a tad more succinct writing-wise.

https://javascript.plainenglish.io/how-to-remove-a-specific-item-from-an-array-in-javascript-a49b108404c shows that for loop with splice is 5x faster than indexOf with splice.

Benchmarking

A cool benchmarking site: https://jsbench.me/nyla6xchf4/1

BigInt

Number to BigInt conversion:

const A = 9007199254740991n
const B = BigInt(9007199254740991)
const C = 1n
const D = C / B

Resources and Links

React: Lifecycle

Can be loosely divided into three primary stages:

Before a Component is mounted – constructor, state initialization, some hooks
When a component is mounted – initial processing logic on props into state (say in the render() method or call itself), componentDidMount() logic to fetch data say, hooks and side-effects
After a Component is mounted – rerendering/updating, prep for unmounting, responding to user inputs

Re-rendering included as needed based on the render operation called. (Consult the detailed Component API reference: https://reactjs.org/docs/react-component.html.) A more specific breakdown:

Mounting – key methods:

constructor()
render()
componentDidMount()

Updating – key methods, rerendering:

shouldComponentUpdate()
render()
componentDidUpdate()
setState()
forceUpdate()

Unmounting – key methods:

componentWillUnmount()

Resources and Links

Code samples:

React: State, Props

Props - immutable, settings, configuration, initialized values, etc. that are passed one-way (unidirectionally) from a wrapping parent element into a child element.
State - the mutable, local Component state.

Props

Consider the following example. A component CustomFooter has several child Components namely CustomLink.

Values are supplied for url and label which are passed from CustomFooter into each CustomLink as Props.

// CustomFooter

import React from 'react'
import CustomLink from '../CustomLink'
import './CustomFooter.css'

export default () =>
    <footer>
        <ul>
            <li><CustomLink url={'https://www.linkedin.com/in/adamintaegerard/'} label={'LinkedIn'}/></li>
            <li><CustomLink url={'https://thoughtscript.io/landing.html'} label={'Thoughtscript.io'}/></li>
        </ul>
    </footer>

CustomLink then accesses the supplied Props through destructuring.

It then uses the supplied values for url and label to populate the href and text values for the rendered Anchor tag:

// CustomLink

import React from 'react'

export default ({url, label}) => <a href={url} rel="nofollow noopener noreferrer" target="_blank">{label}</a>

State

Consider the following example:

AccountSummaries initializes its state Object within the constructor call.
A token is made via makeToken() to authenticate against the endpoint BUDGET_HELPER_API_URL.
An async HTTP GET Request is made against the endpoint BUDGET_HELPER_API_URL.
The Response Object is parsed and checked for a valid Status Code.
The resultant data is then set into State, updating the Component state Object.
Setting or modifying State typically involves the Component re-rendering.
Values in State are destructured const { accounts, authorized } = this.state and made available during Render.

import React from 'react'
import './AccountSummaries.css'
import { asyncGet } from '../../../Helpers/Xhr/Get'
import { BUDGET_HELPER_API_URL } from '../../../Constants'
import { makeToken, parseJson } from '../../../Helpers/Generic'

export class AccountSummaries extends React.Component {
    constructor(props) {
        super(props)
        this.state = {
            accounts: [],
            authorized: true,
            ...this.props
        }
    }

    componentDidMount() {
        try {           
            asyncGet(`${BUDGET_HELPER_API_URL}accounts?auth=${makeToken()}`).then(accounts => {
                if (JSON.parse(accounts).status === 200) {
                        this.setState({
                            accounts: parseJson(accounts),
                            authorized: true
                        })
                    })
                } else {
                    this.setState({
                        authorized: false
                    })
                }
            })
        } catch (ex) {
            console.log(ex)
        }
    }

    render() {
        const { accounts, authorized } = this.state
    }
}

Resources and Links

https://x-team.com/blog/react-reactor-passwordless-spring/

Code samples:

React: Hooks

State

One can now set State within a formerly Stateless Functional Component.

import React, { useState } from 'react'

export function App(props) {
  const [stateVar, setStateVar] = useState(0)

   return (
       <div>
            <button onClick={() => {
              setStateVar(stateVar + 1)
              console.log(stateVar); //0 1 2 3 ...

            }}><code>stateVar</code> Counter</button>
        </div>
  );
}

Note that stateVar must be initializated in useState.

Side Effects

Side Effects are run after a Components renders (hence, side-effect).

import React, { useEffect, useRef } from 'react'

export function App(props) {
    const myRef = useRef(null)

    useEffect(() => {
        myRef.current.style.color = "Red"
        myRef.current.className = "Example"
        console.log("side-effect") // side-effect
        console.log(myRef.current.className) // Example
    })

    return (
        <div>
            <h1 ref={myRef}>My Header</h1>
        </div>
    )
}

Useful to make succinct changes within Stateless Functional Components ("dummy components") post-render using terse/less verbose syntax (no Class constructor initialization, componentDidMount, etc.).

Reducers

Reducers are now supported out of the box and don't have to be associated with an underlying State Provider.

React continue to use this nomenclature to draw comparisons to the prior Map-Reduce naming convention (e.g. - converging sources to a single Reducer).

export const exampleStateInitialization = {
  id: 0,
  text: 'Hi!'
}

export function exampleReducer(state, action) {
  switch (action.type) {
    case 'action_type_a': {
      return {
        ...state,
        id: action.id,
        text: action.text,
      };
    }
    case 'action_type_b': {
      return {
        ...state,
        text: action.text,
      };
    }
        case 'action_type_b': {
      return {
        ...state,
        text: 'LOREM_IPSUM',
      };
    }
    default: {
      throw Error('Unknown action type: ' + action.type);
    }
  }
}

import React, { useReducer } from 'react'
import { exampleStateInitialization, exampleReducer } from './exampleReducer'

export function App(props) {
  // Define a Dispatcher function name to call the Reducer with a specific Event.
  const [exampleReducerState, exampleReducerDispatch] = useReducer(exampleReducer, exampleStateInitialization)
  return (
    <div>
      <h2 onClick = {(e) => {
        exampleReducerDispatch({
          id: 1,
          type: "action_type_b",
          text: "event_text"
        })
     }}>My Button Header</h2>
    </div>
  )
}

Context

import { createContext } from 'react'

// Create and export
export const ExampleContext = createContext("Me, Myself, and I")

import React, { createContext } from 'react'
import { ExampleComponent } from './ExampleComponent'
import { ExampleContext } from './ExampleContext'

// Import and wrap elments
export App(props) =>
  <div>
    <ExampleContext.Provider value={ "You and only you" }>
      <ExampleComponent />
    </ExampleContext.Provider>
  </div>

import React, { useContext } from 'react'
import { ExampleContext } from './ExampleContext'

export function ExampleComponent(props) {
  // Import and use the context  without passing specific Props everywhere and in-between!
  const exampleContextVal = useContext(ExampleContext)

  return (
    <div>
      <h1>{ exampleContextVal }</h1>
    </div>
  )
}

Both updating a React context value and making a React context updateable (from within a sub-Component) involves associating State with the Provider value:

export function App(props) {
  const [example, setExample] = useState("Me, Myself, and I")

  return (
    // Associate the state field here
    <ExampleContext.Provider value={example}>

      <Button onClick={() => { setExample("You and only you") }}>
        I'm Button Text
      </Button>

    </ExampleContext.Provider>
  )
}

Resources and Links

React: Refs

There are two primary ways to create refs (a reference to a DOM Element that's accessible within a React Component):

React.createRef() - primarily used when initializing refs within a React.Component Class constructor:

import React from 'react'

export default class App extends React.Component {
   constructor(props) {
      super(props);
      this.header = React.createRef();
   }

   render() {  
      console.log(this.header) 
      // current: <h1>My Header</h1>

      return (
         <div className="wrapper">
            <h1 ref={this.header} onClick={() => {
               this.header.current.className = 'active'
               this.header.current.innerText = "My New Header"
               this.header.current.style.color = "Red"

               console.log(this.header) 
               // current: <h1 class="active" style="color: red;">My New Header</h1>
            }}>My Header</h1>
         </div>
      )
   }
}

const myRef = useRef(null); - useful to both access a DOM Element that's accessible within a React Component with an initialized value while keeping the value around.

Use with Stateless Functional Components ("dummy components").

import React, {useRef} from 'react';

export default () => {
   const myRef = useRef(null)

   console.log(myRef)
   // current: <h1>My Header</h1>

   return (
      <div>
         <h1 ref={myRef}  onClick = {(e) => {
            myRef.current.innerHTML = "My New Header"
            myRef.current.style.color = "Red"
            console.log(myRef)
            // current: <h1 style="color: red;">My New Header</h1>

         }}>My Header</h1>
      </div>
   )
}

Current

Most refs will expose their attributes after the Component is rendered (e.g. - in a Side Effect or onClick).
Use current to access the current Element State.

Resources and Links

React: Parcel

Build-tool and configuration notes.

npm run build-parcel-prod
npm run build-parcel
cd dist
npx serve

One gotcha: parcel-bundler/parcel#7636. If you add to package.json:

  "engines": {
    "node": "=16.17.0"
  }

You'll get:@parcel/packager-js: External modules are not supported when building for browser. Remove the engines field.

By default, NPX and parcel will serve from: http://localhost:1234/

React: Redux

Overview

Redux provides asynchronous, multi-component, horizontal, State Management for complex single page React apps.

Actions - defines the available operations on state storage.
Container Component - binds Actions to Props. Call the bound Actions within the component to modify the Reducer state.
Reducer - state storage.

Example

Actions:

'use strict'

/**
 *  Default actions for interacting with various Redux stores.
 *
 *  @Author - Adam InTae Gerard - https://www.linkedin.com/in/adamintaegerard/
 */

export const UNSAFE_SAVE = 'UNSAFE_SAVE'
export const REMOVE = 'REMOVE'
export const GET = 'GET'
export const CLEAR = 'CLEAR'
export const SAFE_SAVE = 'SAFE_SAVE'

//Use for public info containing no sensitive information
export const unsafeSave = v => {
  return {type: UNSAFE_SAVE, v}
}

//Use for secure or private info
export const safeSave = v => {
  return {type: SAFE_SAVE, v}
}

export const remove = v => {
  return {type: REMOVE, v}
}

export const get = v => {
  return {type: GET, v}
}

export const clear = v => {
  return {type: CLEAR, v}
}

Stateful Container Component:

'use strict'

/**
 *  Page Container.
 *
 *  @Author - Adam InTae Gerard - https://www.linkedin.com/in/adamintaegerard/
 */

import { connect } from 'react-redux'
import { Page } from './Page'
import { clear, get, remove, safeSave } from '../../../Redux/Shared/DefaultActions'

const mapStateToProps = state => {
  return {
    ...state
  }
}, mapDispatchToProps = dispatch => {
  return {
    save: (key, s) => {
      dispatch(safeSave({data: s, index: key}))
    },
    remove: key => {
      dispatch(remove({index: key}))
    },
    clear: () => {
      dispatch(clear())
    },
    get: key => {
      dispatch(get({index: key}))
    }
  }
}

export const PageContainer = connect(
  mapStateToProps,
  mapDispatchToProps
)(Page)

Reducer:

'use strict'

/**
 *  Encapsulated state storage Reducer.
 *
 *  @Author - Adam InTae Gerard - https://www.linkedin.com/in/adamintaegerard/
 */

import { CLEAR, GET, REMOVE, SAFE_SAVE } from '../Shared/DefaultActions'

let encapsulatedStateObj = {}

const set = (index, data) => {
  encapsulatedStateObj[index] = data
  return Object.assign({}, encapsulatedStateObj)
}, remove = index => {
  delete encapsulatedStateObj[index]
  return Object.assign({}, encapsulatedStateObj)
}, clear = () => {
  for (let i = 0; i < Object.getKeys(encapsulatedStateObj).length; i++) {
    remove(Object.getKeys(encapsulatedStateObj)[i])
  }
  return Object.assign({}, encapsulatedStateObj)
}

/**
 * Default supplied reducer.
 *
 * Caches into state.
 *
 * Partition by key in state.
 *
 * @param state
 * @param action
 * @returns {*}
 * @constructor
 */

export const SafeStorage = (state = encapsulatedStateObj, action) => {
  const type = action['type']
  switch (type) {
    case SAFE_SAVE:
      return set(action['v']['index'], action['v']['data'])
    case REMOVE:
      return remove(action['v']['index'])
    case GET:
      return Object.assign({}, encapsulatedStateObj[action['v']['index']])
    case CLEAR:
      return clear()
    default:
      return state
  }
}

Resources and Links

Code samples:

https://github.com/Thoughtscript/x_team_wp_react/tree/master/xteamClient/reactAppSrc

Angular: Services

Define an @Injectable():

import {Injectable} from '@angular/core'
import { HttpClient } from '@angular/common/http'

@Injectable()
export class EventService {
    constructor(private http: HttpClient) {  }

    getEvents() { return this.http.get('https://localhost:8888/api/events'); }
}

Use it as a dependency:

import { Component, OnInit } from "@angular/core"
import { EventService } from '../../services/event.service'

interface EventResponse {
    status: string
    data: string
}

interface Event {
    uuid: number
    name: string
    msg: string
}

@Component({
    selector: "event",
    template: `
    <main>
        <div id="flex-wrapper">
        <table>
            <tr>
                <th>UUID</th>
                <th>Name</th>
                <th>Message</th>
            </tr>
            <tr *ngFor="let event of events">
                <td>{{event.uuid}}</td>
                <td>{{event.name}}</td>
                <td>{{event.msg}}</td>
            </tr>
        </table>
        </div>
    </main>
  `,
    styles: []
})

export class EventComponent implements OnInit {
    //Make array to hold data
    public events: Event[] = [];

    //Inject the relevant service here
    constructor(private _eventService: EventService) {  }

    ngOnInit() { this.getEvents(); }

    getEvents() {
        // Casting response Objects
        this._eventService.getEvents().subscribe((res: Object) => {
            const E_R = res as EventResponse
            const ALL_EVENTS = JSON.parse(E_R.data) as Event[]
            const SLICED_EVENTS = ALL_EVENTS.slice(0, 5);
            this.events = SLICED_EVENTS;
            console.log(this.events);
        });
    }
}

Export the Module and Component:

import { NgModule } from "@angular/core"
import { CommonModule } from "@angular/common"
import { EventComponent } from "./event.component"

@NgModule({
    imports: [
        CommonModule
    ],
    declarations: [EventComponent],
    exports: [EventComponent]
})

export class EventModule { }

Resources and Links

https://x-team.com/blog/quantum-computation-python-javascript/

Code samples:

Angular: Routing

Provided that a Module and its Components are correctly Exported:

import { NgModule } from "@angular/core"
import { Routes, RouterModule } from "@angular/router"
import { HomeComponent } from "./modules/home/home.component"
import { EventComponent } from "./modules/events/event.component"

const routes: Routes = [
    {
        path: "",
        component: HomeComponent
    },
    {
        path: "event",
        component: EventComponent
    },
];

@NgModule({
    imports: [RouterModule.forRoot(routes)],
    exports: [RouterModule]
})

export class AppRoutingModule {}

Make sure to make Modules visible within the app itself:

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import {HttpClientModule} from "@angular/common/http"

import { AppComponent } from './app.component';
import { EventComponent } from './modules/events/event.component';

import { AppRoutingModule } from "./app.routing";
import { NavModule } from "./modules/nav/nav.module";
import {EventService} from "./services/event.service";

@NgModule({
  declarations: [
    AppComponent,
    EventComponent
  ],
  imports: [
    BrowserModule,
    AppRoutingModule,
    NavModule,
    HttpClientModule
  ],
  providers: [EventService],
  bootstrap: [AppComponent]
})
export class AppModule { }

import {Component} from "@angular/core"

@Component({
    selector: "ng-root",
    template: `
        <div>
            <custom-header></custom-header>
            <router-outlet></router-outlet>
            <custom-footer></custom-footer>
        </div>
    `,
    styles: []
})

export class AppComponent {
}

Resources and Links

https://x-team.com/blog/quantum-computation-python-javascript/

Code samples:

CSS: Specificity

Precedence and Priority

Top to Bottom - the last, bottom-most, entries override previous entries with the same or lesser level of specificity.
- div#divTestId will be blue given the following:
```
/* orange */
div#divTestId {
  color: orange;
}

/* Last entry - blue */
div#divTestId {
  color: blue;
}
```
Specificity - more narrowly defined selectors take greater precedence over less narrowly defined ones.
- div#divTestId is more specific than #divTestId although the two semantically refer to the same element.
- Therefore, #divTestId will remain orange given the following:
```
/* More specific - orange */
div#divTestId {
  color: orange;
}

/* blue */
#divTestId {
  color: blue;
}
```
The !important keyword - overrides the standard Precedence and Priority rules described above.
- Elevates the Priority of an Element such that it can only be overridden by another !important CSS value.

Example

<!-- HTML -->
<div id="divTestId" class="divTestClass">
  <p id="pTestIdOne" class="pTestClass">
    text
  </p>
  <p id="pTestIdTwo" class="pTestClass">
    text
  </p>
</div>

/* CSS */

/* blue */
#divTestId {
  color: blue;
}

/* More specific - orange */
div#divTestId {
  color: orange;
}

/* More specific - green */
#divTestId.divTestClass {
  color: green;
}

/* More specific - pink */
div#divTestId.divTestClass {
  color: pink;
}

/* More specific - red */
div#divTestId.divTestClass p {
  color: red;
}

/* More specific - purple */
div#divTestId.divTestClass > p {
  color: purple;
}

/* less specific - purple */
div#divTestId p {
  color: red;
}

/* less specific - purple */
p#pTestIdTwo {
  color: blue;
}

/* more specific - purple and red*/
div#divTestId p#pTestIdTwo {
  color: red;
}

/* most specific - black and red*/
div#divTestId > p.pTestClass#pTestIdOne {
  color: black;
}

Rendered:

text

CSS: Techniques

Remove default padding and margins:

html, body {
    position: absolute;
    height: 100%;
    width: 100%;
    padding: 0;
    margin: 0;
    top: 0;
    left: 0;
}

Responsive Columns

<!-- HTML -->
<div class="flex-container">
    <div class="item">
        <h5>Example</h5>
    </div>
    <div class="item">
        <h5>Example</h5>
    </div>
        <div class="item">
        <h5>Example</h5>
    </div>
    <div class="item">
        <h5>Example</h5>
    </div>
        <div class="item">
        <h5>Example</h5>
    </div>
    <div class="item">
        <h5>Example</h5>
    </div>
</div>

// SCSS
.flex-container {
  display: flex;
  display: -webkit-flex;
  flex-wrap: wrap;
  width: 100%;

  & > .item {
    text-align: center;
    justify-content: center;
    align-content: center;
    width: 30%;
    border: 2px dashed gray;
    border-radius: 15px;
    padding: 5px;
    margin: 5px;
  }
}

Rendered:

Example

Responsive Centering

<div class="wrapper-example">
  <h1>Example</h1>
</div>

div.wrapper-example {
    display: flex;
    display: -webkit-flex;
    flex-wrap: wrap;
    width: 100%;
}
div.wrapper-example > h1 {
  text-align: center;
  justify-content: center;
  align-content: center;
  width: 100%;
  border: 3px solid black;
  padding: 15px;
  border-radius: 15px;
}

Rendered:

Example

Scroll

::-webkit-scrollbar {
   width: 10px;
}
::-webkit-scrollbar-track {
  background: crimson;
}
::-webkit-scrollbar-thumb {
  background: orangered;
}
::-webkit-scrollbar-thumb:hover {
  background: orangered;
}

Input Text

<input id="exampleTextInput" type="text" placeholder="placeholder_text" />

input[type="text"]#exampleTextInput {
  width: 600px;
  padding: 20px;
  font-size: 20px;
  border-radius: 25px;
  color: turquoise;
  opacity: .55;
  margin: 15px;
}

input[type="text"]#exampleTextInput:focus {
    opacity: .8;
    outline: none;
}

input[type="text"]#exampleTextInput::placeholder {
  color: gray;
}

Rendered:

Markers

<ol>
  <li class="example">Example</li>
  <li class="example">Example</li>
  <li class="example">Example</li>
</ol>

ol > li.example::marker {
    color: orange;
}

Rendered:

Example
Example
Example

Disable Text Selection

.disable-select {
    -webkit-touch-callout: none; /* iOS Safari */
    -webkit-user-select: none; /* Safari */
    -khtml-user-select: none; /* Konqueror */
    -moz-user-select: none; /* Old version of Firefox */
    -ms-user-select: none; /* Internet Explorer or Edge */
    user-select: none; /* All modern browsers */
}

Text Backgrounds

<div>
  I am a <span class="text-background">block of text</span>! Hooray me!
</div>
<div>
  I am another <span class="text-only">
  block of text</span>! Hooray <span class="text">again</span>!
</div>
<div>
  I am a last <span class="text-background border">
  block of text</span>! Hooray <span class="text">again</span>!
</div>

div > span.text-background {
  color: white;
  border-radius: 3px;
  border: 0px solid transparent;
  padding: 1.2px 7px 2px 5px;
  margin: 3px 0px;
  background: #b92b27;
  background: -webkit-linear-gradient(to right, #1565c0, #b92b27);
  background: linear-gradient(to right, #1565c0, #b92b27);
  width: fit-content;
}

div > span.text-background.border {
  border-bottom: 3px solid purple;
}

div > span.text-only {
  background: #b92b27;
  background: -webkit-linear-gradient(to right, #1565c0, #b92b27);
  background: linear-gradient(to right, #1565c0, #b92b27);
  /** Refer to: https://developer.mozilla.org/en-US/docs/Web/CSS/background-clip */
  background-clip: text;
  -webkit-background-clip: text;
  color: transparent;
}

Rendered:

I am a block of text! Hooray me!

I am another block of text! Hooray again!

I am a last block of text! Hooray again!

SVG Optimization

Consult: https://raygun.com/blog/improve-page-load-speed-svg-optimization/

Resources and Links

CSS: Pseudo-Classes

Pseudo-Classes provide additional filtering, querying, and narrowing power to standard Cascading Style Sheet Selectors beyond the innate ability to query for HTML id or class Attributes.

Several Pseudo-Classes involve specifying a distinct state that an Element might find itself in due to a triggering Event. For instance:

:hover - triggered when a user hovers over an Element with their mouse
:focus - triggered when a user clicks into an HTML Input Field

HTML Attribute Querying

Query by HTML Attribute:

[my_attr="my_value"] {
    /* */
}

/* With wildcard - any my_attr value matching my_value */
[my_attr*="my_value"] {
    /* */
}

Refer to: https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors

Some Important Pseudo Classes

/* Triggered when a user hovers over an Anchor element */
a:hover {
    /* */
}

/* First child element of an Unordered List - probably a List element */
ul:first-child {
    /* */
}

/* Last child element of an Unordered List - probably a List element */
ul:last-child {
    /* */
}

Refer to: https://developer.mozilla.org/en-US/docs/Web/CSS/:active

CSS: Media Queries

Media Queries allow one to specify conditional Cascading Style Sheets, styles, or styling without requiring complicated JavaScript.

It's generally a good idea to place narrower Media Query conditions lower to take precedence over less narrowly defined conditions:

@media (max-width: 1850px) {
  #home {
    transform: scale(.70);
  }
}

@media (max-width: 1550px) {
  #home {
    transform: scale(.48);
  }
}

Examples:

max-height only:

@media (max-height: 850px){
    nav#toc-wrapper > ul#toc {
        height: 425px;
    }
}

max-width only:

@media (max-width: 1850px) {
  .container > h2 {
    font-size: 35px;
  }
}

@media (max-width: 550px) {
  .container > block {
    color: red;
  }
}

@media (max-width: 1050px){
    body>div#dune{display:none}
}

Conjoined conditions with Media Types and Media Features:

@media only screen and (orientation: landscape) and (max-width: 6500px) and (max-height: 600px) {
  .container > h2 {
    color: aqua;
  }
  .container > q {
    color: aqua;
  }
}

@media only screen and (max-width: 1000px) {
  .container > q {
    color: green;
  }
}

Refer to: https://developer.mozilla.org/en-US/docs/Web/CSS/@media#media_types for all Media Types and Media Features

Resources and Links

SQL: General Concepts

NOTE: A variety of SQL dialects are used in the examples below (MYSQL, Postgres, MSSQL, and SQLite)

CRUD

CREATE - a resource is persisted and saved.
READ - a persisted resource is scanned, read, or returned in an unmodified way.
UPDATE - a persisted resource is updated.
DELETE - a persisted resource is permanently or softly removed.

Corresponding Basic Operations

CREATE

CREATE statements are typified by the INSERT (Row) or CREATE (Table) keywords.

INSERT INTO PaySchedule (date, name) VALUES ('2023-02-15', 'frank');

DROP TABLE IF EXISTS "Rules";

CREATE TABLE IF NOT EXISTS "Rules" (
    "id"    INTEGER NOT NULL UNIQUE,
    "description"    TEXT,
    "category"    TEXT,
    "rule"    TEXT,
    PRIMARY KEY("id" AUTOINCREMENT)
);

READ

READ statements are typified by the presence of the SELECT keyword.

SELECT * FROM Accounts WHERE active =  1 AND status = 'Active';

SELECT * FROM Hobbies;

SELECT x.newhome + y.newhome + z.newhome + w.newhome + q.newhome
FROM 
  (SELECT SUM(unitcost * quantity) as newhome FROM House WHERE quantity > 0 AND fha = 1) AS x,
  (SELECT SUM(unitcost * quantity) as newhome FROM UpkeepHobbies WHERE quantity > 0 AND fha = 1) AS y,
  (SELECT SUM(unitcost * quantity) as newhome FROM UpkeepClothing WHERE quantity > 0 AND fha = 1) AS z,
  (SELECT SUM(unitcost * quantity) as newhome FROM UpkeepElectronics WHERE quantity > 0 AND fha = 1) AS w,
  (SELECT SUM(unitcost * quantity) as newhome FROM UpkeepKitchen WHERE quantity > 0 AND fha = 1) AS q;

SELECT * FROM MonthlyCosts WHERE choicegroup LIKE "%ALL%";

SELECT COUNT(*) FROM (SELECT grouping FROM MonthlyCostsGroceries GROUP BY grouping);

SELECT y.total / (x.mtbf * 12) AS upkeep FROM 
  (SELECT AVG(mtbf) AS mtbf FROM Upkeep WHERE quantity > 0 AND choicegroup LIKE "%ALL%") AS x, 
  (SELECT SUM(unitcost * quantity) AS total FROM Upkeep WHERE choicegroup LIKE "%ALL%") AS y;

UPDATE

UPDATE statements are typified by the presence of the UPDATE (ROW) or ALTER (TABLE) keyword.

UPDATE Assets SET value = 400 WHERE id = 4 AND account = 8;

UPDATE Demographics SET personal = 750, success = 1, updated = '2020-01-01' WHERE id = 100;

ALTER TABLE example ADD more_text VARCHAR(45);

DELETE

DELETE statements are typified by the presence of the DELETE keyword.

DELETE FROM PaySchedule WHERE date < '2023-02-15' OR date = '2024-02-15';

SQL: ACID

Atomicity guarantees that operations are completely contained in a single unit (Transaction). It either succeeds completely or fails (such that all operations fail – it is not partially successful).

Rollback, database Commit

Consistency guarantees that each Transaction moves the database state from one valid, legal, state to another valid, legal, state.

FK constraints being enforced across two Transactions

Isolation guarantees that concurrent Transactions are disjoint and separate, and that they don’t overwrite each other.

Database READ/WRITE locks

Durability guarantees that persisted data remains persisted across time and system failure/disaster.

Caching systems often lack durability since they often exist only in memory.

SQL: Joins

Retrieving data that's associated by field.

OUTER - matches and NULL values where absent depending on the specifics of the JOIN.

INNER - only what's matched in both Tables.

LEFT, RIGHT, INNER, FULL

LEFT (OUTER) JOIN: Returns all records from the left Table, the matched records from the right Table, and NULL in any Row absent from the right Table .
RIGHT (OUTER) JOIN: Returns all records from the right Table, the matched records from the left Table, and NULL in any Row absent from the left Table.
(INNER) JOIN: Returns records that have matching values in both Tables. Equivalent to: SELECT * FROM table_one, table_two WHERE table_one.id = table_two.id
FULL (OUTER) JOIN: Returns all records when there is a match in either left or right Table, NULL in any Row absent from one of the two Tables.

-- Multi JOIN
SELECT a.name as aname, b.description as name, a.description as adesc, s.name as description, b.value 
FROM Balances AS b 
LEFT JOIN Accounts AS a ON b.account = a.id 
JOIN SummaryTypes AS s ON b.purpose = s.id 
WHERE a.active = 1 AND a.status='Active' AND b.active = 1 
ORDER BY value DESC;

-- INNER JOIN
SELECT * 
FROM example AS e
INNER JOIN employee AS em ON e.id = em.id;

CROSS

Produces the Cartesian Product of two Tables and their Rows.
Every Row in Table A is combined with every Row in Table B.

Self Joins

A JOIN between a Table and itself.

Explicit vs Implicit

Consider the following:

--- Simple JOIN
SELECT * FROM A, B WHERE A.id = B.a_id;

--- Explicit JOIN
SELECT * FROM A JOIN B ON A.id = B.a_id;

The two statements are very nearly functionally identical.
Although there are some differences in terms of the leeway that the Execution Planner has when computing an optimal SQL Plan.

JOIN Order

The order of JOIN statements is typically relevant for any Explicitly Joined statement

SQL optimizers will refine most simple (Implicit) JOINS and have the most leeway in computing an optimal SQL Plan.
SQL optimizers will refine most INNER JOINS through a computed Execution Plan.
However, OUTER JOINS will typically preserve the order specified by the statement.

Generally speaking, remember to keep the leftmost tables to the least number of expected rows.

For instance, if one is joining three Tables A, B, C such that:

The Rows in C are guaranteed to be unique.
The number of Rows likely to be returned by B meeting some condition is far less than the number that'd initially be scanned in A meeting the same.
Then, it's likely far better to join C, B, then A (in that order) to reduce the total number of Rows that need to be compared (in toto) and at each step in the overall JOIN statement.

SQL: Techniques

Some common SQL techniques.

DROP TABLE Check

To create a fresh, empty, new Table drop any existing such Table first OR to prevent accidently overriding an existing Table:

DROP TABLE IF EXISTS "Rules";

CREATE TABLE IF NOT EXISTS "Rules" (
    "id"    INTEGER NOT NULL UNIQUE,
    "description"    TEXT,
    "category"    TEXT,
    "rule"    TEXT,
    PRIMARY KEY("id" AUTOINCREMENT)
);

SELF JOIN

Occassionally, a TABLE will represent multiple entity-kinds (say a PLANT TABLE with ROWS representing both seedlings and their parent plants):

SELECT e1.name AS Plants
FROM Plants AS e1
JOIN Plants AS e2
ON e1.parent = e2.id
AND e1.sproutdate > e2.sproutdate;

Refer to: https://leetcode.com/problems/employees-earning-more-than-their-managers/description/

GROUP BY

GROUP BY is used to condense multiple ROWS with the same field into a single result in the RECORD SET. GROUP BY can be further refined by an accompanying HAVING clause (which acts like a WHERE clause for the specified GROUP). Importantly, in many dialects of SQL, GROUP BY can only be used with Aggregate Functions (ROUND, AVG, COUNT, SUM, MIN, MAX, etc.).

NOTE: HAVING conditions are deferred and apply to the results of a GROUP BY clause and are therefore often faster than their equivalent WHERE clause (which applies the filtering condition against the entire TABLE).

SELECT Name FROM STUDENTS GROUP BY Name, Marks, ID HAVING Marks > 75 ORDER BY Right(Name, 3) ASC, ID ASC;

SELECT e.id, ROUND(AVG(e.account), 2) AS averageAccountBalance
FROM Example AS e
JOIN Examples AS ee
ON e.id = ee.id
GROUP BY e.id, ee.name
HAVING e.account > 750
ORDER BY e.id ASC, averageAccountBalance DESC

NOTE: a double GROUP BY (e.g. - GROUP BY x.id, x.name) can be used to deduplicate associated multi-row RECORD SETS (in say a complex multi-JOIN).

Refer to: https://www.hackerrank.com/challenges/more-than-75-marks/problem

Chained SELECT Statements

SELECT DISTINCT CITY FROM STATION 
    WHERE (CITY LIKE 'A%'
    OR  CITY LIKE 'E%'
    OR  CITY LIKE 'I%'
    OR  CITY LIKE 'O%'
    OR  CITY LIKE 'U%')
    AND CITY IN (

SELECT DISTINCT CITY FROM STATION 
    WHERE CITY LIKE '%a'
    OR  CITY LIKE '%e'
    OR  CITY LIKE '%i'
    OR  CITY LIKE '%o'
    OR  CITY LIKE '%u')

Refer to: https://www.hackerrank.com/challenges/more-than-75-marks/problem

WITH

WITH x AS (
  SELECT months * salary AS earnings, id
  FROM employee
);

SELECT TOP 1 earnings, Count(id)
FROM x
GROUP BY earnings
ORDER BY earnings DESC;

Resources and Links

SQL: NULL

COALESCE

SUM returns NULL not 0 if no values exist that meet the query conditions.
Use COALESCE(SUM(table.column),0)
Also try a different JOIN if a zero summed value fails to appear.
For example, a LEFT JOIN with COALESCE(SUM(table.column),0)

NULL

Use IS NOT NULL vs <> NULL

Postgres: General Concepts

Advantages

Plugin support.
Full support for NoSQL-like JSON in queries, rows, and columns.
Less vulnerable to data corruption (preference for data Consistency/integrity over performance).
dblink now provides remote-database querying/connections.
Non-blocking index creation (CONCURRENTLY).
Postgres Multi-Version Concurrency Control (MVCC) reading never block writing and vice-versa. Also, see the article on ACID.

Disadvantages

Deprecated: No cross-database querying (a decisive factor for many database systems at scale: MySQL was a top choice for that reason) prior to 8.2.
Deprecated: Slightly slower than MySQL (using the older MyISAM engine - a decisive factor for many database systems at scale: MySQL was a top choice for that reason) for READ and transaction-heavy workloads.

Resources and Links

Postgres: Indexes and Views

Indexes - used to improve query performance within a Table - Indexes a Table by one or more Columns.
Views - a logical representation of a Table or Tables.

Indexes

Postgres supports:

Hash indexes and B-Tree indexes
Partial Indexes support conditioned indexing: CREATE INDEX CONCURRENTLY my_index ON my_table (column1_name) WHERE amount > 0;
Concurrent non-blocking indexing: CREATE INDEX CONCURRENTLY my_index ON my_table (column1_name, column2_name);

Implicit Indexes are automatically created for any Primary Key on a Table by default.

Materialized Views

Essentially a cached Table that stores the results of a query:

DROP MATERIALIZED VIEW my_view_name;

CREATE MATERIALIZED VIEW my_view_name
AS
    SELECT * FROM example;
    --- Assume column 'name'

That can be queried itself:

SELECT * FROM my_view_name;

And that can be refreshed:

REFRESH MATERIALIZED VIEW my_view_name;

Concurrent refresh:

CREATE UNIQUE INDEX my_index ON my_view_name (name);

REFRESH MATERIALIZED VIEW CONCURRENTLY my_view_name;

Resources and Links

Postgres: JSON

Postgres supports both JSON and JSONB data types:

JSON is stored in a standard String format.
JSONB is a more performant format that uses binary (hence, the "b") to improve indexing and querying at the expense of more complex Serialization.

Operators

Given a table:

id	json_col	json_array_col	jsonb_col	jsonb_array_col
`1`	`"[1,2,3]"`	`"[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]"`	`[1,2,3]b`	`[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]b`

-> - allows one to query into a JSON field shared by all rows in a column (say json_col). E.g. - jsonb_col -> 'name'
->> - extracts the JSON value at the specified index (numeric) or the value at the specified key. E.g. - json_col ->> 2
::int, ::json, etc. - since JSON fields lack a Postgres type, use :: to cast the value to a type.

Examples

To initialize an example Postgres table:

DROP TABLE IF EXISTS example;

CREATE TABLE example (
  id INT,
  json_col JSON,
  json_array_col JSON,
  jsonb_col JSONB,
  jsonb_array_col JSONB
);

-- Insert values into table.

INSERT INTO example VALUES (1,
  '[1,2,3]'::json,
  '[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]'::json,
  '[1,2,3]'::jsonb,
  '[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]'::jsonb
);

Use the following queries to retrieve the desired JSON data:

-- queries

SELECT * FROM example;

-- insert via json

INSERT INTO example VALUES (2,
  '[1,2,3]'::json,
  '[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]'::json,
  '[1,2,3]'::jsonb,
  '[{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]'::jsonb
);

INSERT INTO example
SELECT id, json_col, json_array_col, jsonb_col, jsonb_array_col
FROM json_populate_record (NULL::example,
    '{
      "id": 3,
      "json_col": {"name": "bob", "age": 111},
      "json_array_col": [{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}],
      "jsonb_col": {"name": "sarah", "age": 2222},
      "jsonb_array_col": [{"id": 0, "name": "a"},{"id": 1, "name": "a"},{"id": 2, "name": "c"}]
    }'
);

-- query into json array

SELECT arr -> 'id' AS json_id, arr -> 'name' AS json_name
FROM example e, json_array_elements(e.json_array_col) arr
WHERE (arr ->> 'id')::int > -1;

-- query json column

SELECT json_col::json ->> 2 FROM example;

SELECT json_col -> 'age' FROM example;

SELECT json_col -> 'age' AS json_age FROM example WHERE (json_col ->> 'age')::int = 111;

-- query into jsonb array

SELECT arr -> 'id' AS json_id, arr -> 'name' AS json_name
FROM example e, jsonb_array_elements(e.jsonb_array_col) arr
WHERE (arr ->> 'id')::int > -1;

-- query jsonb column

SELECT jsonb_col::json ->> 2 FROM example;

SELECT jsonb_col -> 'age' FROM example;

SELECT jsonb_col -> 'name' AS jsonb_name, jsonb_col -> 'age' AS jsonb_age
FROM example WHERE (jsonb_col ->> 'name') = 'sarah';

Resources and Links

Code samples:

https://github.com/Thoughtscript/postgres_json_practice

Kubernetes: Install Minikube

I've chosen minikube here since it's the Kubernetes distribution that's used during the CKA exam and probably the easiest to get set up locally. It's a bit underpowered for Enterprisey stuff but suffices for these purposes.

Gotcha's

You must have docker installed for the typical (and default) minikube installation.
To install minikube in a cloud VM you'll need at least 2 CPU cores (Medium size on AWS).

Minikube Installation

Basically follow this: https://minikube.sigs.k8s.io/docs/start/

Download the correct version after installing docker.

# Verify install 
minikube version

# Start minikube
minikube start

Note that minikube offers a simple kubectl wrapper: minikube kubectl. This is, for all intents and purposes, just an automated installer for the correct version of kubectl. Otherwise, most people recommend just using kubectl directly (it saves time typing too).
Note further that minikube is both a tool and a local cluster with several namespaces.

Minikube Removal

Some relevant commands:

# Stop your local cluster
minikube stop

# Removes your local minikube cluster (and start over)
minikube delete

# Removes all local clusters
minikube delete --all
minikube delete --purge

Kubectl

Installing kubectl directly is likely a test question: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

Kubernetes: Kubectl Inspection

Key operation: kubectl get.
Display one or many resources.
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get

minikube will install the default namespace automatically.

# Display all namespace.
kubectl get namespaces

# Display all pods in every namespace.
kubectl get po --all-namespaces

# Display all pods in the `default` namespace (will be empty by default).
kubectl get po -n default

# Display all services in every namespace.
kubectl get services --all-namespaces

# Display detailed information about test
kubectl describe namespace test

Generally speaking, kubectl get ENTITY defines a valid operation where ENTITY is one of these.

Kubernetes: Main Entities

I divide Kubernetes entities into two kinds:

Meta-level, organization, groupings.
Primitive entities.

Groupings

Grouping entities. These group and help manage Pods and Nodes.

Namespace

https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
A Namespace is a handle for grouping one or more Clusters.
This allows Clusters to be categorized, grouped, etc. by role.

Cluster

https://kubernetes.io/docs/concepts/architecture/
Clusters are divided into Nodes. Clusters are groupings of Nodes.
Clusters are not created like other entities.
Instead, Clusters are managed by Namespaces.

Service

https://kubernetes.io/docs/concepts/services-networking/service/
An abstract way to expose one or more Pods as a networking service.
Essentially, a Service assigns a group of Pods a shared IP Address and/or DNS name.

Basic

Basic entities.

Node

https://kubernetes.io/docs/concepts/architecture/nodes/
A Node may be a virtual or physical machine.
Multiple Pods can run in a Node.
Nodes contain additional components (resources) to run Pods.

Pod

https://kubernetes.io/docs/concepts/workloads/pods/
A group of one or more Containers.
The smallest unit of deployment.

Container

https://kubernetes.io/docs/concepts/containers/
Containers are Images that may be built (compiled, or otherwise) and run.
They represent an Image at runtime, its state, etc.
Conceptually, this is akin to a Docker Container (and may in fact be identical in some cases since Kubernetes can operate using Docker).
Note, existing Containers cannot be added to a Kubernetes Pod since they are immutable.

Image

https://kubernetes.io/docs/concepts/containers/images/
Container Images represent an encapsulated resource with all its dependencies.
This is akin to a Docker Image (and may in fact be identical in some cases since Kubernetes can operate using Docker).

Deployment

A Kubernetes Deployment essentially wraps a Docker Image with its required startup and configuration information within a YAML file (or equivalent).

Kubernetes: Kubectl Create

Key operation: kubectl create.
Display one or many resources.
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#create

# Create namespace test
kubectl create namespace test

# Display test
kubectl get namespaces

# Set namespace context
kubectl config set-context --current --namespace=test

Kubernetes: Kubectl Run

Key operation: kubectl run.
Create and run a particular Image in a Pod.
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#run

Kubernetes: Kubectl Docker

Discussion and example highlighting major differences between using Docker Images in Docker and using them in Kubernetes.

Docker

Terminal One:

# Download image
docker pull elixir

# Open an interactive terminal through Elixir in Docker
## Note that calling this without -it will not allow commands to be passed to iex
docker run -it elixir

Terminal Two (Admin Console - use as needed)

# Display the container and obtain the CONTAINER ID
docker stats

# Display the downloaded images and obtain the IMAGE ID
docker images --all

# Remove container
docker stop CONTAINER_ID
docker rm CONTAINER_ID

# Remove downloaded image
docker rmi -f IMAGE_ID

# Remove all containers and volumes
docker system prune

# Remove all containers and volumes
docker system prune --volumes

So, in the above, one is able to pull elixir and run bash terminal.

Kubernetes

Close the other terminals.
Check out this awesome blog: https://www.yeahshecodes.com/docker/run-docker-images-locally-with-minikube
So, from what I can gather using docker pull directly through kubectl results in CrashLoopBackOff (at least when using minikube). In other words, don't kubectl run elixir --image=elixir --namespace=test.
In fact, it seems many Docker Images won't run in minikube - at least following the same steps below for elixir:latest, postgres, alpine, and ubuntu only resulted in errors. This suggests that more careful configuration is required (probably creating a Deployment resource file in YAML).

Instead, use a docker-compose.yaml file:

 version: "3.9"
 services:
   nginx:
     image: localtest:v0.0.1
     build: .
     ports:
       - "80:80"

and, a dockerfile:

 FROM nginx:latest
 EXPOSE 80

Terminal Three:

# Ensure both minikube and docker are in the same env
eval $(minikube docker-env)

# Build from dockerfile
docker-compose up -d

# View docker images (from within minikube!)
docker images --all

# Create namespace test
kubectl create namespace test

# Set namespace context
kubectl config set-context --current --namespace=test

# Review all resources in namespace
kubectl --namespace test get all

# Deploy the Docker image as a Pod to the Container
kubectl run localtest --image=localtest:v0.0.1 --image-pull-policy=Never --namespace test

# Review all resources in namespace
kubectl --namespace test get all

Docker: Basic Commands

# Build from dockerfile
## Use this over `docker build - < Dockerfile`
## Note that the dockerfile copies in ANY files in this directory
docker build .

# Docker metrics and processes
docker images --all
docker info
## Get the CONTAINER ID <aa9f01c38d04>
docker stats

# Cleanup
## Remove image
docker rmi -f IMAGE_ID
## Remove container
docker rm CONTAINER_NAME
docker stop CONTAINER_NAME
docker system prune --volumes

Refer to: https://github.com/Thoughtscript/docker

Also: https://github.com/Thoughtscript/postgres_json_practice/blob/master/1%20-%20dockerfile/docker.sh

Resources and Links

Code samples:

Docker: dockerfile

FROM postgres:13.0

# Execute init scripts
## These only have to be copied into /docker-entrypoint-initdb.d/
COPY init_json_sql.sql /docker-entrypoint-initdb.d/

Useful Dockerfile Commands

RUN specifies a Bash Command or Expression to be executed.
VOLUME defines a Volume Mount. (Note: These can be Bind Mountings that are transient and confined to the local Container.)
COPY simplified and cp-equivalent instruction.

Resources and Links

https://docs.docker.com/engine/reference/builder/

Git Cheat Sheet

Some common and useful Git commands.

Checkout New Branch From Origin

git fetch origin my_branch
git pull
git checkout my_branch

View Branch Commit History

git log

View Entire Local Change History

git reflog

View File Changes

By SHA:

git diff --name-only af43c41d..HEAD
git diff af43c41d..master

By branch:

git diff --name-only origin/deploy..master
git diff origin/deploy..master

Correct Previous Commits

Review, alter, remove, amend last 3 commits:

git rebase -i HEAD~3
Type i to enter interactive mode.
Find the line with the desired commit hash. Modify it using pick, drop, etc.
Hit the esc button to exit interactive mode.
Type wq to save and close the file (Git will proceed through the stipulated changes) or type q! to close the file abandoning all changes.
git push -f to override previous changes - do not use this on master/main only ever within a development branch.

Git Amend

Correct the last commit message:

git commit --amend -m "Your new message"

Discard Uncommitted Branch Charge

git clean -f -d
git reset --hard HEAD

Abandon a Rebase

git rebase --abort
git clean -f -d
git reset --hard HEAD

Change Branch W/ Same Name As Dir

If a branch contains a dir in the root with the same name as a branch, Git will now complain.

Use the following instead:

git fetch origin deploy (if freshly cloned)
git switch -f deploy

Set Environment Config

git config --global user.username "my_username"
git config --global user.email "my_email@email.com"

Resources and Links

Engineering: HTTP and SSL

REST - Representational State Transfer (read below).
HTTP - Hypertext Transfer Protocol - often equated with REST. The basis for the World Wide Web typified by Request/Response Objects, Headers, Parameters, HTTP Methods, and Sessions.
SSL - TLS - HTTPS - Hypertext Transfer Protocol with Security.

Certificates

Certificate Authority - validates a Certificate (say, X.509).
Public and Private Keys - a kind of Symmetric Encryption that bifurcates Credentials into two pieces.

REST

REST (Representational State Transfer) - a client-server architectural paradigm that encourages the following core precepts:

Client-Server - the familiar Client to Server paradigm. A Client connects to a Server.
Statelessness - user Session data isn't stored or persisted between Sessions.
Layered System - read Service Layer Architecture article.
Uniform Interface - common data types, HTTP Methods, etc.

Methods:

OPTIONS - indicates that a Client is seeking a Server and further HTTP Request information (required HTTP Headers, etc.).
GET - indicates that a Client will READ a persisted resource.
POST - indicates that a Client will CREATE a persisted resource.
PUT - indicates that a Client will UPDATE a persisted resource.
PATCH - indicates that a Client partially UPDATE a persisted resource.
DELETE - indicates that a Client intends to DELETE a persisted resource.

Request / Response Lifecycle

The HTTP Request/Response Lifecycle:

Request Object -
1. Headers - specifies the Content-Type, CORS information, Credentials or Tokens, etc. (sent back from the Web Client).
2. HTTP Method - OPTIONS, GET, POST, PUT, PATCH, DELETE .
3. URL (Uniform Resource Locater) - the IP Address with optional Port number, DNS address, etc.
4. Parameters (optional) - specify query filters to narrow down some query result: id, page, etc.
5. Body (optional) - submitted or requested information can be regimented as JSON, text, a form, etc., and encapsulated by the Request and Response Objects.
Response Object -
1. Headers - specifies the Content-Type, CORS information, Credentials or Tokens, etc. of the Response Object (sent back from the Web Server).
2. Body (optional) - submitted or requested information can be regimented as JSON, text, a form, etc., and encapsulated by the Request and Response Objects.

Engineering: TCP, UDP, IP, DNS

IP - Internet Protocol Address

IPv6
- Hexadecimal
- 128-Bit IP Address
- Eight groups of four hexadecimal digits
- Example: 2031:0ca7:75×3:13f3:0103:8b1e:0310:7532
IPv4
- Purely numeric (decimal)
- 32-Bit IP Address
- Four octets in the inclusive range 0-255
- Example: 127.0.0.1

CIDR - Classless Inter-Domain Routing

Solution for allocating IP Addresses and IP Routing.
The convention <MY_IP_ADDRESS>/<CIDR_LENGTH> (e.g. - 10.0.0.0/16) specifies the CIDR Length for the IP Address (Prefix).
The CIDR Length defines the number of bits reserved for the Netmask.
- The Netmask divides an IP Address into sections akin to dividing a Zip Code into Street Addresses.
- Used to specify which part of the Host IP Address space is reserved for the Host number and which is reserved the Subnet number.
- The lower the CIDR Length, the more IP Addresses are available within the network.
Classes and their respective Netmasks:
- Class A - 255.0.0.0
- Class B - 255.255.0.0
- Class C - 255.255.255.0

CIDR Notation	Host Formula	Available Hosts
`/8`	`232-8 - 2`	`16,777,214`
`/9`	`232-9 - 2`	`8,388,606`
`/10`	`232-10 - 2`	`4,194,302`
`/11`	`232-11 - 2`	`2,097,150`
`/12`	`232-12 - 2`	`1,048,574`
`/13`	`232-13 - 2`	`524,286`
`/14`	`232-14 - 2`	`262,142`
`/15`	`232-15 - 2`	`131,070`
`/16`	`232-16 - 2`	`65,534`
`/17`	`232-17 - 2`	`32,766`
`/18`	`232-18 - 2`	`16,382`
`/19`	`232-19 - 2`	`8,190`
`/20`	`232-20 - 2`	`4,094`
`/21`	`232-21 - 2`	`2,046`
`/22`	`232-22 - 2`	`1,022`
`/23`	`232-23 - 2`	`510`
`/24`	`232-24 - 2`	`254`
`/25`	`232-25 - 2`	`126`
`/26`	`232-26 - 2`	`62`
`/27`	`232-27 - 2`	`30`
`/28`	`232-28 - 2`	`14`
`/29`	`232-29 - 2`	`6`
`/30`	`232-30 - 2`	`2`

(From: https://erikberg.com/notes/networks.html)

DNS - Domain Name Service - associates a (purely) numeric IP Address (127.0.0.1) with a human readable/friendly Domain Name (localhost).
TCP - Transmission Control Protocol - a Transport Layer protocol that complements IP (TCP/IP). Is used to negotiate and determine connections themselves prior to data being transmitted.
UDP - User Datagram Protocol - responsible for actually sending packets and messages over TCP/IP. One-directional, requires some other system to actually manage the connections themselves. (Although it can be used without establishing or verifying connections.)
HTTP/2 - introduces Huffman Encoding to compress packet sizes when initially negotiating an HTTP Request.
QUIC - HTTP/3 - built on top of UDP and removes TCP as the Transport Layer and optimizes some of the initial handshaking resulting in reduced packet sizes, I/O, etc. Multiplexed, persistent connection.
A Proxy is used as an intermediary between an IP Address and a destination resource.
- Note: Proxying typically refers to Forward Proxying.
- Forward Proxying (a Forward Proxy Server) is connected to by a Client in order to mask the Client IP Address. The Forward Proxy then connects to a desired resource. (The Proxy masks the incoming IP Address and uses the desired outgoing IP Address.)
- A Reverse Proxy is a hidden Gateway or intermediary between the Client and desired resource. (AWS API Gateway is paradigmatic - a Client makes an HTTP Request to a specific endpoint; the endpoint is associated with a configured Lambda Function which is called on behalf of the inbound Request.)
- Forward Proxying is deliberately used or known to the Client. By contrast, in Reverse Proxying scenarios the Proxy acts unbeknownst to the Client.
- Note that both kinds of Proxying involve the same sequential pattern: a Client C connects to an intermediary Proxy P to access a desired target resource R.
An SSH Bastion serves as an intermediary between an incoming SSH connection and a desired resource.
- The SSH Bastion serves to "jump" the incoming SSH connection to the desired resource after the inbound connection is authenticated and authorized.
- In this way, destination resources are protected from being exposed publically and all inbound SSH requests can be verified/controlled.

Resources and Links

Engineering: Status Codes

200 - Status OK: OK.
201 - Created successfully typically via POST: Created.
202 - Queued, batched successfully but not complete: Accepted.
301 - URL permanently moved
400 - Bad request - malformed: Bad Request
401 - Unauthorized, lacking credentials: Unauthorized.
403 - Forbidden, no authorization to access even with credentials: Forbidden.
404 - Page or resource can't be found at specified URL: Page Not Found.
405 - Wrong method
500 - Internal Server Error

Engineering: Common Port Numbers

443 - HTTPS/TLS default port number
8080 - HTTP/URL default port number
22 - SSH default port number
25 - SMTP default port number
1433 - MSSQL default port number

Note: 80/443 are usually the default port numbers used by gRPC since it uses HTTP/S - for example as in this article I wrote: https://goteleport.com/docs/api/getting-started/

Engineering: Naming Conventions

Some common naming conventions:

Dry - database naming convention - Person table with a Person Class domain Object Relational Mapping (ORM)
Snake Case - some_person
Camel Case - somePerson
Pascal - Person

Engineering: Basic Terminal Commands

Kill Task

Taskkill /IM node.exe /F

killall node

Open SSL

openssl genrsa -out key.pem 2048
openssl req -new -sha256 -key key.pem -out csr.csr
openssl req -x509 -sha256 -days 365 -key key.pem -in csr.csr -out certificate.pem

Grant Permission

Grant (all) Read, Write, and modify permissions:

sudo chmod +rwx file

SSH Keys

Generate a new SSH Public and Private Key pair using ssh-keygen:

ssh-keygen -t ed25519 -C "your_email@example.com"

Open the Public Key (with suffix .pub) and copy the Public Key into the necessary cloud resource account (GitHub, etc.).

Copy the Private Key into ~/.ssh (or equivalent) and associate with ssh-agent:

ssh-add ~/.ssh/id_ed25519

SSH Key mappings will be visible in /.ssh/known_hosts.

An excellent resource: https://docs.github.com/en/authentication/connecting-to-github-with-ssh

Resources and Links

Engineering: Mac Install Notes

Personal notes to get a development environment setup and configured on macOS 10.15.3.

Note: configuring .bash_profile is required for macOS-specific directory paths.

Note: configuring .bashrc is required for many Bash-specific paths.

Note: configuring .zshrc is required for many Zsh-specific paths (macOS Catalina+).

The notes below are written for Bash. In most cases, the same commands and export statements can be used interchangeably between Bash and Zsh (provided you use the correct configuration file).

I'm also not a huge fan of Homebrew (think Catalina system directory structure incompatibility, multiple lingering installs, version conflicts that typically or eventually arise locally, etc.) so the installation steps below tend to prefer the native installers wherever possible and appropriate.

Xcode

For developing Apple mobile, desktop, tablet, and watch, applications.

Swift installation:

Use this link: https://apps.apple.com/us/app/xcode/id497799835 to get Swift + Xcode
Test the installation using $ swift --version
Test the Xcode installation using $ xcodebuild -version

CocoaPods:

Requires Ruby gem manager
Execute the following: $ sudo gem install cocoapods
Use $ pod install to download the dependencies into your project
NOTE: You must open the <PROJECT_NAME>.xcworkspace file rather than the <PROJECT_NAME>.xcodeproj file to build and compile your project (with Pods) correctly

Python 2.7

For Python, PIP, and Django apps.

Python 2.7 installation:

Python 2.7 is pre-installed on macOS 10.x.
Test the installation using $ python --version

PIP installation:

Execute $ sudo easy_install pip
Test the installation using $ pip --version

Django installation:

Execute $ sudo pip install Django==3.0.3
Alternatively, execute $ sudo pip install -r requirements.txt

Python 3

For Python 3.x.

Python 3.x installation:

Download from: https://www.python.org/downloads/mac-osx/
Test the installation using $ python3 --version

PIP:

PIP is automatically installed as part of the Python 3 installation
Upgrade PIP: $ python3 -m pip install --upgrade pip
Install dependencies: $ python3 -m pip install -r requirements.txt
List all installed libraries: $ pip freeze
Clear out PIP cache: $ pip uninstall -y -r <(pip freeze)

Venv:

Venv is automatically installed as part of the Python 3 installation
Create a Venv environment: $ python3 -m venv VENV_ENV
... and activate it: $ source VENV_ENV/Scripts/activate

Ruby

For Ruby on Rails apps.

Ruby installation:

Ruby 2.6.3p62 is pre-installed on macOS 10.x.
Test the installation using $ ruby --version
Test Ruby gem manager using $ gem -v

Rails installation:

$ gem install rails

C++

C++ GCC 4.2.1 is distributed with Xcode 11.3 (whose installation instructions are provided above).
Test the installation using $ gcc --version

CMake:

Download the CMake Unix/Linux Source (has \n line feeds) from: https://cmake.org/download/
Extract the contents and execute $ bash bootstrap within the root directory
Then execute $ cd Bootstrap.cmk and $ bash make
Copy the following into .bash_profile using $ sudo nano ~/.bash_profile (and modify as needed):

export PATH=$PATH:/Users/USER_NAME/Desktop/cmake-3.16.6/bin

Test the installation: $ cmake --version

Java

For Java Spring, Gradle, Maven, and Tomcat stacks.

Java installation:

Use this link: https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Test the installation using $ javac -version

Apache Tomcat 8.5.x installation:

Download the .zip from http://tomcat.apache.org
Copy the extracted directory to your desktop (or some other appropriate location)
Navigate to the ROOT/bin directory and execute the following Bash command sudo chmod +x *.sh
Execute $ sudo bash startup.sh to launch Tomcat on the default port localhost:8080

Refer to the very helpful: https://wolfpaulus.com/tomcat/ for more comprehensive configurations.

Gradle installation:

Download from: https://gradle.org/install/
Copy the following into .bash_profile using $ sudo nano ~/.bash_profile (and modify as needed):

export PATH=$PATH:/Users/USER_NAME/Desktop/gradle-6.2/bin/

Test the installation using $ gradle --version

Node

For NodeJS server and client apps.

NVM installation:

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.1/install.sh | bash
$ sudo touch ~/.bash_profile
$ sudo touch ~/.bashrc
$ sudo nano ~/.bash_profile - copy the contents below into this file
$ sudo nano ~/.bashrc - copy the contents below into this file

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"

(Copy the above into configuration files.)

Test the installation using $ nvm ls
Download the desired version of Node using $ nvm install 10.0.0 && nvm use 10.0.0

Typescript installation:

Execute npm install -g typescript

Golang

Go to: https://golang.org/dl/ and download the newest version
Install Go using the downloaded installer
Test the installation using $ go version

Rust

For Rust apps.

Rust installation:

Execute $ sudo curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Test the installation using $ rustc --version

Rust uninstallation:

$ rustup self uninstall

Refer to the Rust documentation.

Disable DS_Store Files

Open Terminal App:

Located in Applications > Utilities

Enter the following command:

defaults write com.apple.desktopservices DSDontWriteNetworkStores true

Consult: https://www.techrepublic.com/article/how-to-disable-the-creation-of-dsstore-files-for-mac-users-folders/

Brew

As of macOS 14.4.1 (23E224) you may need to install Homebrew for certain commonly used dependencies:

Download Homebrew or by running:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Run the commands:

(echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> /Users/YOUR_USERNAME/.zprofile

eval "$(/opt/homebrew/bin/brew shellenv)"

to add Homebrew to PATH.

Note that .zprofile is the default shell config from the official documentation.

Run source .zprofile (or source .bash_profile) to reload the relevant terminal configs
To install pkg-config on Mac: brew install pkg-config (required for Python3)

zsh

Duplicate any .bash_profile entries into .zprofile (or .zshrc):

.zprofile is for login shells
.zshrc is for interactive shells

Consult: https://unix.stackexchange.com/questions/71253/what-should-shouldnt-go-in-zshenv-zshrc-zlogin-zprofile-zlogout

Engineering: Windows Install Notes

It's been a while since I used a Windows machine for development. (Windows has traditionally been my preferred and most commonly encountered development environment of choice.) Have to say, I'm immensely impressed by Windows 11!

There are many great changes overall (and, I disagree with some critics - the new taskbar is much improved - centering the icons is a much better experience than having to peer over in the left corner on large screens/UHD 4K TVs).

Node

This will install Node, Git, and Python 2.7.

As of 5/21/2022 - Windows 11 Pro 21H2 22000.675.

For Node:

Make sure to open your terminal of choice with the Run As Administrator option (equivalent in some ways to sudo).
Download Git SCM
Download NVM
Download Python 2.7
Rename the python executable to to python2 (node-gyp requires this nomenclature).
Search locally for sysdm.cpl to open System Properties -> Advanced -> Environmental Variables in Windows 11.
Add the install path to your User and System Variables.
nvm install X.Y.Z && nvm use X.Y.Z (in Bash or ZSH) for the specific version of Node you want.
I've had the best luck using Visual Studio 2017 (Community) rather than a newer version. Download after signing into Microsoft here.
Make sure to tick off:
- C# and Visual Basic Rosyln compilers
- MSBuild
- Static analysis tools
- Visual Studio C++ core features
- Windows 10 SDK (10.0.17763.0)
Run npm config set msvs_version 2017
Run npm i or whatever npm or node commands you desire.

Java

This will install Java 1.18, Maven, Tomcat 10 and Gradle.

Updated 8/25/2022 - Windows 11 Pro 21H2 22000.856.

Download Java 1.18
Download Maven 3.8.6
Download Tomcat 10
Download Gradle 7.5.1
Add the relevant System variables under Advanced system settings > Environment Variables.
GRADLE_HOME should point at your unzipped Gradle root directory.
JAVA_HOME should point to your Java root directory.
MAVEN_HOME should point to your unzipped Maven root directory.
Then add the following to your Path.
%JAVA_HOME%\bin,
%GRADLE_HOME%\bin
%MAVEN_HOME%\bin

Run the folloiwng commands to verify your installs.

javac -version
mvn -version
gradle --version

Navigate to: http://localhost:8080/ after running the new(ish) Apache Tomcat10 executable.

Engineering: Azure VM Ubuntu 18.04

Brief setup notes for Azure VM Ubuntu 18.04 on Mac OSx.

Create Public and Private Keys

Create a private key. A public key will be created automatically for you.

ssh-keygen -t rsa

Make note of your private key password - this is used to authenticate below.

After creating your Azure Ubuntu 18.04 VM. Take note of your Public IP.

Ensure that the default SSH port is left open.

Connect from Local Machine

sudo ssh -i path/private_key user@public_ip

Use your private key password after connecting.

Ubuntu 18.04 VM

Slightly different setup than Ubuntu 14.04.

sudo apt-get update
sudo apt-get upgrade

Git

sudo apt-get update
sudo apt install git

Node

sudo apt install nodejs
sudo apt install npm

Java

Note: It's recommended to download Java 11+ directly from Oracle.

It's further recommended to use the OpenJDK 11.0.2 (and avoid other options).

Refer to: https://jdk.java.net/archive/

sudo apt-get update
wget "https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz"
sudo tar -xzvf openjdk-11.0.2_linux-x64_bin.tar.gz
sudo mv jdk-11.0.2 /usr/lib/jvm/

# Config
sudo nano /etc/environment

# Add the line below
# JAVA_HOME="/usr/lib/jvm/jdk-11.0.2/"

# Config
sudo nano ~/.bashrc
# Add the lines below
# JAVA_HOME=/usr/lib/jvm/jdk-11.0.2/
# PATH=$JAVA_HOME/bin:$PATH
source ~/.bashrc

# Verify
echo $JAVA_HOME
javac --version

Tomcat:

groupadd tomcat
useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat
cd /opt/ 
sudo wget https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.2/bin/apache-tomcat-9.0.2.tar.gz
tar -xzvf apache-tomcat-9.0.2.tar.gz
sudo mv apache-tomcat-9.0.2 tomcat

Then:

# Permissions
sudo chown -hR tomcat:tomcat tomcat
sudo chmod +xr tomcat/bin/

# Config
sudo nano ~/.bashrc

# Add the line below
# CATALINA_HOME=/opt/tomcat

# Config
source ~/.bashrc

# Verify
echo $CATALINA_HOME

You can now access /opt/tomcat/bin/ to execute: sudo bash startup.sh.

Your Tomcat server will be available by default on http://YOUR_IP:8080 (note the lack of https here).

Engineering: OWASP

Summary of common top 10 OWASP identified security vulnerabilities:

Broken Access Control
- Violation of The Principle of Least Privilege
- Privilege elevation
- CORS misconfiguration
- Insecure direct object references
- Metadata manipulation
Cryptographic Failures
- Weak, old, or deprecated cryptographic algorithms
- Lack of encryption: TLS, etc.
- Low randomness used: pseudorandom, improper seed
Injection
- SQL Injection
- Query Injection
- Lacking character escaping, validation, filtering, or sanitization
- Ability for users to execute queries from a client input
Insecure Design
- Ability of malicious actors to exploit weaknesses in system design, architecture, or business logic
- Example: not restricting the number of tickets a person can buy, not having rate limiting, etc.
Security Misconfiguration
- Leaving Ports open
- Insufficient security hardening
- A system whose dependencies aren't updated
- Disabled security features
- Incompatible dependency versions
Vulnerable and Outdated Components
- Deprecated, out-of-date, or old software and/or dependencies
- Lacking a critical security patch
Identification and Authentication Failures
- Lacks multi-factor authentication
- Allows for weak passwords
- Transmission of passwords in plaintext
- Susceptibility to automated or brute-forcing attacks
Software and Data Integrity Failures
- Downloading unsigned dependencies from a remote repository
- Downloading dependencies from an untrusted remote repository
- A compromised update
- Insecure deserialization
Security Logging and Monitoring Failures
- Exposing critical or sensitive data within accessible logs
- Unclear or unhelpful logging messages
- Lacking proper logging for critical events
- Lacking the proper visibility into critical events, services, or systems
- Lacking appropriate mechanisms to remediate problems: escalation, on-call rotations, etc.
Server-Side Request Forgery
- Improperly allowing a web application and/or its resources to be accessed or controlled remotely
- Allowing an unauthenticated, unvalidated, or unauthorized agent to access a resource remotely

Resources and Links

https://owasp.org/www-project-top-ten/

Engineering: Application Security

A concise, quick, overview.

SOC2

Every completed task must be documented.
Every customer-facing or production task must have a corresponding ticket.

OWASP

Available here: https://owasp.org/www-project-top-ten/
Common Security Vulnerabilities include XSS, SQL Injection, String Timing attackes, etc.

Encrypt at Rest

Encrypt database information within the database.
Never store sensitive PII information in plaintext.

Ports

Verify that all unneeded Ports are closed.

Secure Access Workstation

Increasingly popular.
An air-gapped physical machine allowing access to a single user at a single precise time.
The air-gapped physical machine then connects to an SSH Bastion.

Engineering: OAuth 2.0

Grant Types

The four main Authorization Grant Types are as follows (note that the PKCE extension is now recommended for supplemental use in all other Grant Types):

Authorization Code

The flow:

A resource R1 requests access to a protected resource R2.
A user (who typically owns or manages both resources) is directed to an intermediary authentication server to authorize R1’s getting R2’s data (for some duration of time).
After authorizing access, the user is redirected to a specified URL (often a view in R1): the Redirect URL.
The most common and secure Grant Type. Use this by default.
The paradigm or model OAuth flow that inspired the others.

Client Credentials

Used in server-to-server access where one resource is able to access a second (common in micro-services).

Device Code

Where a device with limited human input capabilities is granted access to a resource through another human input medium.
Used to give IoT devices access to some endpoint. These devices may lack a keyboard.

Refresh

Used to refresh a previously acquired valid token.

Deprecated

Note that the Password and Implicit Grant Types are now deprecated:

Password

Deprecated.
Essentially, your standard user-submitted password to get a token authentication scheme.

Implicit

Deprecated.
Often used in minimal security conditions (like a simple JavaScript app) to acquire a token from an auth server through a (registered) Client ID.
Tokens are not refreshed but simply reacquired.

Resources and Links

https://oauth.net/2/

Engineering: Service Layer Architecture

Presentation Layer - what the user sees and interacts with. A web Client, mobile device, terminal, etc. Corresponds to the View in MVC pattern.
Service Layer - contains Business Logic, Controllers, Handlers, and Services that process Data being sent from a Client, and that return processed Data to the Client.
Data Layer - persisted Data that exists at rest or used in-memory.

Engineering: Model View Controller

Model - programmatic representations of Data defined in a Domain. Run Time, in-memory, and programmatic representations of persisted data that may or may not be managed 1:1 through an Object Relational Mapping framework like Hibernate.
View - what's rendered and presented to the user.
Controller - HTTP REST Controllers, Handlers that reside at sepecifc Endpoints (e.g. - URL, URL Path, IP Address, Domain Name, Port Number) etc. to handle specific HTTP Requests / HTTP Methods.

Engineering: Design Patterns

Some common design patterns.

Singleton

Refer to the other Singleton article.

Factory

The Factory design pattern creates copies of a specific type. The Factory itself is usually instantiated once, often a Singleton (single instance object).

// Java
public class Dood {
    //...
}

public Dood doodFactory() {
    //...
    getDooder() {
        return new Dood();
    }
}

Abstract Factory

Suppose that K is a subclass of T: make a Factory of kind T that returns a Factory or makes Objects of kind K (through abstract implementation.

The Abstract Factory design pattern is essentially the Factory design pattern but involves Abstract Factories in correspondence with and supporting an abstract (here, meaning higher-order) interface implementation hierarchy or abstract class hierarchy:

First, a (higher-order) interface:

// Java
public interface A {
    //...
}

Two implementations of that interface:

// Java
public class B implements A {
    //...
}

public class C implements A {
    //...
}

Now, we create a factory hierarchy to parallel the implementation hierarchy:

// Java
public interface AbstractFactory {
    public A createA();
}

We then create two further factories concrete implementations of AbstractFactory*`:

// Java
public class BFactory implements AbstractFactory {
    public A createA() {
        B b = new B();
        return b;
    }
}

public class CFactory implements AbstractFactory {
    public A createA() {
        C c = new C();
        return c;
    }
}

Model View Controller

Refer to the other Model View Controller article.

Decorator

Uses annotations like @Component to configure some item or add functionality.

Examples:

Angular
Java Spring
Java EE

Adapter

The Adapter pattern takes two incompatible implementations, Interfaces, or Classes, and provides a way to bridge them. (Hence, "Adapter" - to adapt them to each other.)

// Java
public interface GermanPlugConnector {
    public void giveElectricity();
}

public class GermanElectricalSocket {
    public void plugIn(GermanPlugConnector plug) {
        plug.giveElectricity();
    }
}

// Java
public interface UKPlugConnector {
    public void provideElectricity();
}

public class UKElectricalSocket {
    public void plugIn(UKPlugConnector plug) {
        plug.provideElectricity();
    }
}

These are thus far incompatible and require an Adapter to bring them into harmony:

// Java
public class GermanConnectorToUKSocketAdapter implements UKPlugConnector {
    private GermanPlugConnector plug;

    public GermanConnectorToUKSocketAdapter(GermanPlugConnector plug) {
        this.plug = plug;
    }

    @Override
    public void provideElectricity() {
        plug.giveElectricity();
    }
}

Above we implemented the compatible Plug connector (UKPlugConnector) but overrode provideElectricity() so that it now invokes giveElectricity() on the incompatible Plug connector (GermanPlugConnector).

We have thus performed a little "switch-a-roo" on the main point of incompatibility by using a third interface to bring the two incompatible types into harmony thereby.

Now we explicitly invoke our adapter to representationally allow the GermanPlugConnector to plug into the UKElectricalSocket:

// Java
GermanPlugConnector plugConnector = //.. create a GermanPlugConnector

//Create a UKElectricalSocket
UKElectricalSocket electricalSocket = new UKElectricalSocket();

//We adapt the two
UKPlugConnector ukAdapter = new GermanConnectorToUKSocket(plugConnector);

//And now receive the electricity
electricalSocket.plugIn(ukAdapter);

Example corrected, updated, and modified from: http://www.vogella.com/tutorials/DesignPatternAdapter/article.html

Resources and Links

http://www.journaldev.com/1827/java-design-patterns-example-tutorial

Engineering: The Twelve-Factor App

Summary of Twelve-Factor App Principles:

Codebase - One codebase tracked in revision control, many deploys.
- A single codebase is used for an App.
  - E.g. - a specific repo in GitLab, BitBucker, Subversions, Azure Team Foundation, or GitHub.
- Use a versioning system to track different deployments and changes.
- Use a version control tool like Git or Subversion.
Dependencies - Explicitly declare and isolate dependencies.
- All dependencies are explicitly and expressly declared in a manifest.
- Examples: package.json, pom.xml, requirements.txt, etc.
- Use a package manager.
  - Dependencies should be versioned and modularized so their exact contents can be specified.
Config - Store config in the environment.
- App Configuration is kept with the App.
- It should be stored as Environmental Variables within the Environment itself.
- Example: passing Docker parameters that are exposed as Environmental Variables within an App Container.
- App Configuration should be grouped by (Staging) Environment.
Backing Services - Treat backing services as attached resources
- Any Service or infrastructure dependency is treated as a Resource that's attached (containerized or configured along with the Service in question).
- Examples: docker-compose.yml, Terraform, Cloud Formation
Build, Release, Run - Strictly separate build and run stages.
- Separate a deployment into Build, Release, and Run stages.
- All Releases should have a unique identifier.
Processes - Execute the app as one or more stateless processes.
- Apps should be run as Processes.
- They should be Stateless.
- Example: containerizing and App in a Docker Container that spins up and initializations the App state each time it's run (state isn't preserved).
Port Binding - Export services via port binding.
- Preference for directly binding an App to a Port via a Webserver library (Jetty).
- As opposed to using a Web Container (Tomcat).
Concurrency - Scale out via the process model.
- While one may need to spawn or multithread, these processes should use established tools and infrastructure to do so (Node exec, JVM Threads, Windows Process management).
- Don't create/spawn Daemons when the above can be used out of the box.
Disposability - Maximize robustness with fast startup and graceful shutdown.
- Processes should be easy to terminate.
- Deployed Services shouldn't be unnecessarily dependent on or entangled with other systems. They should be sufficiently decoupled so that they can started and stopped easily.
Dev/Prod Parity - Keep development, staging, and production as similar as possible.
- (Staging) Environments should be 1:1.
Logs - Treat logs as event streams.
- All logging events are printed to STDOUT and never routed within the app.
- All log organization is handled at the Enivronment level.
- Example:
  - Set Logging Levels within an AWS Environment so that it can be collated into a log ingestion Service.
  - Rather than routing logs using distinct say, Log4J LoggingAppenders.
Admin Processes - Run admin/management tasks as one-off processes.
- They should have a finite execution interval (should not be ongoing or occasion indefinite access to say PROD).
- They should always use the most up-to-date config and code (not ad-hoc scripts or old code versions).

Resources and Links

https://12factor.net/

Engineering: Producers and Consumers

Consumer - The resource that consumes, receives, and/or uses Messages or Events sent by the Producer.
Producer - The emitter or sender of a Message or Event to the Consumer

Several commonly encountered patterns that involve Producer and Consumer topics:

Publish-Subscribe
- Subscribers listen to, wait for, poll, or otherwise directly subscribe to a Publisher (specifically, a Publisher's Topics) which emits an Event or Message to all appropriate Subscribers.
- Examples:
  - AWS Simple Notification Service (AWS SNS)
  - Apache Kafka
  - WebSockets
Message Queue
- A Message Queue sits in-between a Producer and Consumer. Often explicitly uses an intermediary called (or appropriately likened to) a Message Broker.
- Examples:
  - ActiveMQ
  - RabbitMQ
  - AWS Simple Queue Service (AWS SQS)
SOAP
- A Consumer requests a WSDL from the Producer to mirror the Domain entities of a Producer (generates the shared contract for the Consumer).
- This allows SOAP requests to be correctly made from the Consumer to the Producer and for the Consumer to correctly handle the corresponding SOAP responses.
REST
- A Client (Consumer) makes an HTTP Request to a Server (Producer) which returns an HTTP Response.

Note that this distinction isn't strictly mutually exclusive. While Quastor refers to RabbitMQ as thoroughly Pub-Sub, the actual RabbitMQ documentation refers to it as Publish/Subscribe (since it can be configured as such).

Resources and Links

Code samples:

Engineering: Systems Design

Topics

Separation of Concerns - Access, Visibility, proper decomposition of a Monolithic app, database access, subnets, private networks
Concurrency - Thread, Thread and Worker Pooling, database Connection Pooling, use of Mutex and Thread-Safety, Asynchronous and Non-Blocking, Load Balancing, Sharding
Events, Messaging, and Event Priority - use of Cron Jobs, Queues, Event Scheduling, Event Workflows
Fault Tolerance - High Availability and Disaster Recovery, ProdEng manual overrides, data auditing and integrity, Graceful and Exponential Backoff, Dead Letter and Retry Queues
Performance - Caching, Views and Indexes, Algorithm optimization, Elasticity, duplex connections
Best Practices - Security, compliance, Integration Testing and environments, End to End Acceptance Tests, etc.

Some Approaches

Some common, famous, and or high-performance approaches:

Decouple Events, Consumers, and Producers - Apache Kafka remains a top choice at many firms (PayPal, Uber, Dosh, etc.).
A move from Relational Databases to Document Store Databases reducing overhead from Sharding, need for Cross-Database JOINS, SQL Table Normalization, and IO/latency.
- Modifying SQL Table Columns is very expensive on large datasets that are distributed over many Shards.
Extensive use of Sharding, Consistent Hashing (keeps Hash invariant as number of Shards increases), and Load Balancing across different Application Layers.
Lots of Caching:
- Clear distinction between READ and WRITE intensive resources (and corresponding resource responsibilities - READ Replica, etc.).
- Materialized Views, etc.
- Extensive use of Memcached, Redis, and/or Couchbase for READ intensive operations.

AWS: Certified Cloud Practitioner

Some last notes I took before taking and passing the AWS Certified Cloud Practitioner Exam (February 22, 2022).

Key areas I wanted to focus on and understand better.

Databases

DynamoDB - Unstructured NoSQL, auto-scales
Aurora - Cloud-first MySQL and Postgres replacement, self-healing, Aurora is more performant, durable, scalable, resilient than RDS
Redshift - data warehouse
RDS - Managed DB, more DB’s supported than Aurora (Oracle)

AWS Billing and Cost Management Tools

By order of information:

AWS Cost Explorer - query resource cost via API, visual UI, the highest level of granularity
AWS Cost Reports - generates S3 reports
AWS Budget - predict spending, optimize use, some forecasting
AWS data migration tools

By order of max data to transfer:

Snowcone - GB to TB
Snowball Edge
Snowmobile - TB to PB

AWS IAM distinctions

Policy - an object that defines an identity’s permissions
Role - groupings of policy that facilitates a specific set of responsibilities
User
Group

AWS Gateway differences

API - Allows access to API endpoints, methods
Internet - VPC to public internet, bidirectional
NAT - resources in VPC to public internet, unidirectional
File, Storage - optimizes multipart uploads and bandwidth for file uploading

AWS Identity Management services

Cognito vs AWS SSO - Access to Apps, Services vs. Access Across AWS Accounts

Note that AWS SSO has been deprecated and replaced with AWS IAM Identity Center.

Different AWS security services

AWS Inspector - Finds and identifies security vulnerabilities and security best practices, EC2
AWS Trusted Advisor - AWS best practices (general), an AWS Support service
AWS Security Hub - Integrates with Trusted Advisor, finds and recommends improvements to security practices
AWS GuardDuty - Threat analysis on logs

App/Resource security:

AWS Shield - DDOS
AWS WAF - web app exploits
AWS Network Firewall - inbound, outbound rules

Keys/licenses:

Secrets Manager - App secrets, DB credentials
KMS, CloudHMS - generates and signs cryptographic keys - ERC20, SSL, Web Server identity verification
Artifact - Compliance
IAM - Permissions
Certificate Manager - TLS

AWS network security differences

Network ACL - applies to VPC
Network Security - applies to instances
AWS Network Firewall - applies to networks

Response Times

Business
- < 4 hours production system impaired
- < 1-hour production system down
Enterprise
- < 4 hours production system impaired
- < 1-hour production system down
- < 15-minute business critical
- Also, only one that has Technical Account Manager
- Concierge support

Resources and Links

AWS SAA-C03: Overview

Some notes I took before taking the AWS Certified Solutions Architect - Associate Exam.

Key areas I wanted to focus on and understand better.

Conventions

I'll use the stylistic format AWS <SERVICE_NAME> to indicate an AWS Service rather than a feature of that Service.

Test Topics

Test Topics and some of their associated services.

Domain 1: Design Secure Architectures - 30%
- AWS IAM
- AWS Control Tower
- AWS KMS
- AWS Cognite
- AWS Guard Duty
- AWS Macie
- AWS Shield
- AWS WAF
- AWS Secrets Manager
- AWS VPC
- AWS Storage Services
Domain 2: Design Resilient Architectures - 26%
- AWS SQS
- AWS Secrets Manager
- AWS SNS
- AWS Fargate
- AWS Lambda
- AWS API Gateway
- AWS Transfer Gateway
- ALB
- AWS Route 53
Domain 3: Design High-Performing Architectures - 24%
- AWS S3
- AWS Batch
- AWS Athena
- AWS Lake Formation
- AWS Storage Gateway
- Amazon Kinesis
- AWS CloudFront
- AWS DirectConnect
- AWS VPN
- AWS EFS
- AWS EBS
- AWS Elasticachae
- AWS Data Sync
- AWS Glue
- AWS EMR
Domain 4: Design Cost-Optimized Architectures - 20%
- AWS Cost Explorer
- AWS Cost Reports
- AWS Budget

High Availability

AWS Regions and Availability Zones

Typically 3 (max 6) Availability Zones per AWS Region

Replication

Replication across multiple Availability Zones is synchronous
Read-replicas (AWS Aurora, AWS RDS, etc.) follow asynchronous replication

Disaster Recovery

Multi-Site - A multi-site solution that runs in AWS (in parallel in the cloud) along with your exsiting on-site infrastructure.
Warm Standby - A scaled-down version of the environment is always running.
Pilot Light - A minimal version of an environment is always running.
Backup and Restore - Data is backup and used as a restore point.

Resources and Links

AWS SAA-C03: IAM

Some finer distinctions

Policy - An object that defines an identity’s permissions
- Can be attached to Groups
- Can be grouped into Roles
Role - Groupings of policies that facilitate a specific set of responsibilities
Groups - Only contain Users, not Groups
Users - Can belong to no Group or many Groups
Root - Created by default and used to manage the AWS IAM account itself
- Super user
- Should create and use Admin users over the course of normal administration

Kinds of IAM Policies

Identity-Based - granted to an Identity (User)
Resource-Based - granted to an AWS Resource (like an AWS Lambda Function)
Session-Based - granted programmatically, by session when an IAM Role is assumed

Policy Evaluation Logic

Evaluation Factors

Identity-Based policies
Resource-Based policies
IAM Permission Boundaries - sets the maximum permission an Identity-based policy can grant to an IAM User or IAM Role
AWS Organizations Service Control Policies (SCPs) - defined at the AWS Organization level
Session policies - programmatically supplied

Precedence

In order of precedence:

An explicit Deny
An Allow within a Service Control Policy
- If not, implicitly Deny
An Allow granted to a Resource and by an associated Resource-Based Policy
An Allow granted to an Identity and by an associated Identity-Based Policy
- If not, implicitly Deny
An Allow granted within a Permissions Boundary
- If not, implicitly Deny
An Allow granted to a Session Principal: (a) with a Session Policy or (b) within a Role Session
- If not, implicitly Deny

AWS Organizations

An account management service that consolidates multiple AWS Accounts into a higher, top-level, organizational unit.
Consolidated Billing for all associated/grouped Accounts.
Global, cross-regional.

AWS Directory Services

AWS Managed Microsoft Active Directory.

AWS Control Tower

Simplifies and standardizes the setup and governance of AWS multi-account environments.
Extends AWS Organizations.

Resources and Links

https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html#policy-eval-basics

AWS SAA-C03: Security

General AWS security.

Tokens

AWS STS

AWS Session Token Service

Provides temporary credentials for an AWS Account or IAM User.
One-time use or until the token expires
- Can be better for granting temporary permissions than setting an IAM Policy or assuming a Role

Secrets

AWS KMS

AWS Key Management Service

AWS KMS manages encryption Keys.
KMS Keys
- Symmetric AES-256
- Asymmetric RSA, ECC
- Multi-region Keys
  - Managed independently although they can be used interchangeably.
  - Destroying one does not destroy the others.
Integreates with most AWS Services.

AWS SSM

Secure store for configuration and secrets.
Optional AWS KMS integration.

AWS Secrets Manager

Newer service for storing secrets.
Can configure forced rotation of secrets every specified number of days.
Using AWS KMS to encrypt secrets.

TLS and SSL

AWS Certificate Manager

Manage, deploy, and provision TLS and SSL certificates.
Supports both public and private certificates.
Supports automatic certificate renewal.
Integrates with:
- AWS API Gateway
- Application Load Balancers

Doesn't integrate with AWS EC2.

Firewalls

AWS WAF

For protecting web apps from common web exploits (Layer 7, HTTP)
Deployed on:
- Application Load Balancers
- AWS API Gateways
- AWS CloudFront
- AWS APPSync GraphQL API
- AWS Cognito User Pool
Define Web Access Control Lists (ACLs):
- HTTP Method
- IP Address
- Geo-region

AWS Firewall Manager

Manage rules for all AWS Accounts in an AWS Organization
Common sets of security rules for:
- AWS WAF
- AWS Shield Advanced
- AWS EC2 Security Groups
- AWS Network Firewall (VPC)
- AWS Route 53 Resolver DNS Firewall

DDoS

AWS Shield

Distributed Denial of Service (DDoS) protection.
AWS Shield Standard - Free.
AWS Shield Advanced - $3,000/month per AWS Organization.

Automated Detection

AWS GuardDuty

Intelligent thread discovery to protect AWS Accounts.
Uses Machine Learning to discover anomaly, 3rd party data.
Sources data from:
- AWS CloudTrail Event Logs
- VPC Flow Logs
- DNS Logs
- Kubernetes Audit Logs
Can define AWS EventBridge Rules to trigger on findings.

AWS Inspector

Automated security assessments for EC2 Instances, container images, Lambda Functions.

AWS Macie

Machine Learning and pattern matching service to detect sensitive data in AWS.
Identifies PII.

Active Directory

AWS AWS Directory Service

AWS Directory Service for Microsoft Active Directory - specific to Microsoft AD
- AWS Managed Microsoft Active Directory (AD)
- Fully-managed by AWS
Integrates with AWS IAM

Federated Services

Allows multiple identity providers to be combined into a single authentication and authorization process.
Allows multiple identity managment systems to be interoperable.
Allows other trusted identity managmenet systems to verify the identity of a user for the others.

AWS SAA-C03: Monitoring

AWS CloudWatch

Monitoring, logging, metrics, alarm

CloudWatch Alarms

Associate with Log Filter Expressions, Metrics.
Trigger based on certain conditions or states.
Composite Alarms monitor multiple other Alarms.

CloudWatch Logs

Log Groups - represents an application.
Log Streams - specific containers, application instances, etc.
Filter Expression - can query across Log Events and trigger Alarms.
Can define Expiration Policies.

CloudWatch Metrics

Use prebuilt or define customized Metrics to associate with Alarms, dashboards.
Belong to CloudWatch Namespaces.
Timestamped

Unified CloudWatch Agent

Deployed onto a AWS EC2 Instance
Used to observe customized metrics (like on Instance CPU use) and send them to AWS CloudWatch

AWS Event Bridge

Schedule Cron Jobs.
Or define reactive rules to respond to a service doing something.
Integrates with most other AWS services.

AWS Cloud Trail

Provides governance, compliance, and auditing for AWS Accounts.
Trace API calls made within an AWS Account across multiple services.

Cloud Trail Events:

Management Events - track operations performed on AWS Resources.
Data Events - Lambda invocation events, API calls, etc.
Insight Events - detect unusual activity, anomalous patterns.

AWS Config

Used to assess, audit, and evaluate the configurations of AWS resources.

Config Rules - evaulate the config settings of AWS resources.
- Managed Rules - predefined.
- Customized Rules - customized by user.
Config Remediation - automatically remediates non-compliant resources such as AWS EC2 and AWS S3.
Config Notifications - notifies when the configuration of monitored resources change.

AWS SAA-C03: CloudFront

AWS CloudFront Price Classes

In order by included regions.

Price Class All - all regions, best performance.
Price Class 200 - most regions but excludes the most expensive regions.
Price Class 100 - only the least expensive regions.

AWS CloudFront Features

Geo-Restriction - restrict by AWS Region.
Integrates with AWS WAF.
Cache Invalidation - set a Time to Live (TTL) and automatically delete files from the cache you're serving from.

AWS Global Accelerator

Uses the AWS internal network to route applications.
Uses Edge Locations to send traffic to your app.
Uses Anycast IP which is created for your app.
- All servers hold the same IP Address.
- A client is routed to the nearest one.

AWS Global Accelerator is usually a better option than Route 53 Geoproximity Routing for large, globally distributed, apps.

AWS SAA-C03: Networking

VPC

Virtual Private Cloud

5 allowed per AWS Region
The maximum Classless Inter-Domain Routing (CIDR) per AWS VPC is 5:
- Min size is /28 (16 IP Addresses)
- Max size is /16 (65536 IP Addresses)
- (The lower the CIDR Length, the more IP Addresses are available.)
Only Private IPv4 Address ranges are allowed:
- 10.0.0.0 - 10.255.255.255 (10.0.0.0/8)
- 172.16.0.0 - 172.31.255.255 (172.16.0.0/12)
- 192.168.0.0 - 192.168.255.255 (192.168.0.0/16)
Example:
- A VPC is defined with IPv4 CIDR block: 10.0.0.0/24:
  - The VPC supports 256 IP Addresses.
  - One can then define two subnets within the VPC each with 128 IP Addresses.

VPC CIDRs should not use IP Addresses that overlap.

Refer to: https://stackoverflow.com/a/56834387 and IP Addresses.

And: https://docs.aws.amazon.com/vpc/latest/userguide/subnet-sizing.html

Also: https://www.rfc-editor.org/info/rfc1918

VPC Subnet

AWS reserves 5 IP (IPv4) Addresses in each Subnet. For example, given CIDR block 10.0.0.0/24:

10.0.0.0 would be reserved as the Network Address.
10.0.0.1 would be reserved for the VPC router.
10.0.0.2 would be reserved for mapping to the Amazon-provided DNS.
10.0.0.3 is reserved for future use.
10.0.0.255 - the Network Broad Address is not supported so AWS reserves this (to prevent it from being used).

VPC Peering

Privately connects two VPCs using AWS' own internal network.
Connected VPCs behave as if they are the same network.
Overlapping CIDRs shouldn't be used in any of the connected networks.

Endpoints

So-called Private Links.
Allows one to connect AWS Services using a Private Network rather than over the Public Internet.
Consider the scernario where an AWS Service (say AWS S3) must be connected to from within a Private VPC.
- One would define an Private Endpoint and/or Gateway Endpoint and connect without going through the Public Internet.

Flow Logs

Captures all information about network traffic:
- VPC Flow Logs
- Subnet Flow Logs
- Elastic Network Interface Flow Logs
Used to troubleshoot connectivity issues.

Traffic Mirroring

Duplicate network traffic/requests so they can be sent to security appliances.
Used to capture and inspect network traffic within a VPC.
Monitor, troubleshoot, inspect connectivity, security, and traffic.

Network Security

Network and VPC-specific security.

Bastion Host

SSH Bastion (Jump) Host.
Configuration:
- Bastion Host Security Group: Allow the Inbound Port 22 on a restricted CIDR (say, the public CIDR being used).
  - This allows authenticated persons to connect using SSH for further verification.
- EC2 Instance Security Groups: Allow the Inbound Private IP of the Bastion Host (or its Security Group)
  - Allows the Bastion Host to jump to the EC2 Instances

NAT Instance

Network Address Translation

Allows EC2 Instances in Private Subnets to connect to the internet.
Requirements:
- Must be launched in a Public Subnet.
- Must disable EC2 setting: Source / destination Check.
- Must have an Elastic IP attached to it.

Deprecated but still tested for in the exam apparently.

NACL

Network Access Control List

Controls traffic from and to Subnets.

Network Firewall

Protects a VPC.
From Layer 3 to Layer 7 protection.

Remote Access

Site to Site VPN

A fully-managed Virtual Private Network (VPN).
Creates a secure connection between an on-premises VPN and an AWS VPC in the cloud.

Direct Connect

Provides a dedicated private connection from a remote network to a VPC.

Gateways

Used to connect networks (and often for Remote Access scenarios).

Virtual Private Gateway

Used to facilitate a Site-to-Site VPN connection.
Attached to the VPC one will be connecting a VPN to.

Customer (Device) Gateway

A physical device that connects a physical, remote, network to an AWS VPC in the cloud.

Transit Gateway

Used to simplify complex network topologies.
Cross-regional connections.
Can peer Transit Gateways across AWS Regions.
Examples:
- Hub-and-Spoke (star) topology connecting 6 VPCs across 4 AWS Regions.
- Connecting 3 VPCs (A,B,C) so that A is connected to B and B is connected to C but not A to C or vice-versa.

Internet Gateway

Define Route Tables.
Specifies routing for inbound and outbound traffic.

NAT Gateway

Connects EC2 Instances in a Private Subnet to a Public Subnet.
Deployed in a Public Subnet with Private Subet Route Tables updated to point internet-bound traffic to the NAT Gateway.

Resources and Links

AWS SAA-C03: Route53

A Domain Registrar
Handles typical DNS attributes:
- A - maps to IPv4
- AAAA - maps to IPv6
- CNAME - maps Hostname to another Hostname
- NS - specify Name Servers for DNS resolution
Handles record settings:
- TTL
- Routing/forwarding

Public vs Private

Private Hosted Zone
- Records that determine how to route traffic in a VPC
Public Hosted Zone
- Records that determine how to route traffic on the internet

Routing

Geolocation - route by user location
Weighting
- Controls the percentage of requests and traffic that go to a specific resource or URL
- Assign by relative weight
Failover - route to a backup location

Health Checks

AWS Route 53 Health Checks can be configured to monitor:

Endpoints - are associated with AWS Data Centers.
- AWS Route 53 will periodically ping so-configured Endpoints.
Other Health Checks
- Called a Calculated Health Check.
- A compound, combined, or complex Health Check.
Cloud Watch Alarms and the underlying Metrics that are used to configure that Alarm.
- Will source its data from the underlying Metrics .
- Or, from an Alarm Data Stream (used to calculate the state of the Alarm).

Furthermore:

A Health Check can transmit its Status to an (another) Alarm as the data source for that Alarm.
Health Checks can trigger Alarms.
Health Checks can send notifications (via AWS SNS).

Comparing Kinds of Health Checks

Consider an EC2 Auto-Scaling Group vs an ALB Health Check:

ALB Health Check - ALB Health Check type
- Cannot be an EC2 Health Check
EC2 Auto-Scaling Group - EC2 Health Check type
- Can use ALB Health Check type
So, an ALB might terminate an EC2 Instance when an ASG does not.

Resources and Links

AWS SAA-C03: Messaging

AWS Simple Queue Service
AWS Simple Notification Service
AWS Kinesis

AWS SQS

Producer's send Messages to a (FIFO) Queue that Consumer's Poll
Default retention: 4 Days (maximum 14 Days)
Used to decouple Application Tiers
SQS scales automatically
An event in an SQS Queue is typically processed by one Consumer (e.g. - with Visibility Timeouts)

Queue Types

SQS Standard Queue
SQS Dead Letter Queue
SQS Delay Queue
SQS FIFO Queue

Batch

Up to 10 messages can be processed at once.

Visibility Timeout

Immediately after a message is received, it remains in the queue
Visibility Timeouts prevent other Consumers from processing the message agin
The default is 30 seconds

Polling

Short Polling
- Occurs repeatedly in short time-frames
- Queries only a subset of SQS servers
Long Polling
- Queries are made every 0-20 seconds (exclusive-inclusive)
- Queries the entire set of SQS servers
- Is AWS-recommended since it's less costly and more accurate

AWS SNS

Event Producer's send Messages to one SNS Topic
Event Receivers subscribe to an SNS Topic
The SNS Topic will broacast Messages to all Receivers
- An event is sent to and processed by all *Receivers

AWS Kinesis

Refer to Data

Collect, process, and analyze streaming data in real-time
IoT Telemetry
Kinesis Data Streams
Kinesis Data Firehose
Kinesis Data Analytics
Kinesis Video Streams

Partition Keys

AWS MQ

Specific to MQTT

AWS SAA-C03: Data Migration

AWS Snow Family

A device is provided to submit data to AWS directly (and physically) without using one's network.

Snowcone - up to Terrabytes
- HD - 8 TB of HDD Storage
- SSD - 14 TB of SSD Storage
Snowball Edge - up to Petabytes
- Storage Optimized - 80 TB
- Compute Optimize - 42 TB
Snomobile - up to Exabytes
- Typically, < 100 PB
- A physical semi-truck arrives and allows one to transfer up to 1 EB

AWS Edge Computing

Snowcone
Snowball Edge

AWS Transfer Family

Used for file transfers in and out of S3

Supports FTP, FTPS, SFTP
Managed infrastructure
Integrates with Active Directory, LDAP, Okta, AWS Cognito, etc.

AWS Migration Services

AWS App Migration - migrate a full application stack
AWS Database Migration - migrate databases

AWS DataSync

NFS, SMB, HDFS, S3
On-premises to cloud requires an agent to move data to and from
Syncs data using S3, EFS, and FSx

AWS Storage Gateway

Connects on-premises data and cloud data.
Typically used to backup data.
Types
- S3 File Gateway
  - NFS and SMB
  - Integrates with Active Directory
- FSx File Gateway
  - AWS access for Windows File Server
  - SMB, NTFS, and Active Directory
- Volume Gateway
  - iSCSI backed by S3
  - Backed by EBS Snapshots
- Tape Gateway
  - For physical tape drives
  - iSCI
  - Virtual Tape Library backed by S3

AWS FSx

FSx for Windows
FSx for Lustre
- High performance computing
- Machine learning
- Linux cluster
FSx File System
- Scratch: temporary is fast but impermanent
- Persistent: data is persisted within the same Availability Zone
FSX for NetAPP

AWS SAA-C03: Other Data Services

Other AWS tools to process, ingest, query, store, and analyze data.

SQL Based

AWS Athena

Serverless service to query and analyze S3 data.
Supports CSV, JSON, ORC, Avro, and Parquet.

AWS Redshift

Based on PostgresSQL for Big Data analytics.
Query on multiple data sources.
Faster than Athena due to indexing.

AWS OpenSearch

Successor to ElasticSearch.
Security through Cognito, IAM, KMS encryption, TLS.

AWS EMR

Elastic MapReduce

Helps to provision and configure Hadoop.
Bundled with Apache Spark, HBase, Presto, Flink.
Composed of up to hundreds of EC2 Instances.

AWS QuickSight

Serverless machine learning, interactive dashboards.
For business analytics, visualizations, business insights, ad-hoc analysis.
In-memory SPICE engine for imported data.

AWS Glue Based

AWS Glue

Convert data into Parquet format as part of an ETL (Extract, Transform, Load) pipeline.
Converts CSV for use in Lambda Functions or AWS Athena.
Catalog of datasets.

AWS Lake Formation

Date Lake: a central place to store your data.
Clean, transform, discover, and ingest data into your Data Lake.
- Combine structured and unstructured data in your Data Lake.
Built on AWS Glue.
- With out-of-box blueprints for S3, RDS, Relational and NoSQL databases

AWS Kinesis Based

Refer to Messaging

Kinesis Data Streams are used to collect and process large streams of data records in real time.

Typically real-time data analytics.
Logs, metrics, and reporting.
Complex stream processing.
Can be combined or linked into Directed Acyclic Graphs.

Kinesis Data Firehose is used to stream data into Data Lakes, warehouses, and analytics services.

Used to load massive data sets.
Can load massive data sets from hundreds of thousands of sources that needn't be real-time data streams.

AWS Kinesis Data Analytics

Real-time analytics on Kinesis Data Streams and Firehose.

AWS Managed Streaming for Kafka

AWS Managed Streaming for Apache Kafka (AWS MSK):

Alternative to AWS Kinesis.
MSK creats and manages Kafka Broker and Zookeeper Nodes (in earlier versions of Kafka).
Data is stored in AWS EBS Volumes for indefinite periods of time.
Has a serverless mode.

AWS SAA-C03: Databases

Choose:

RDBMS
- AWS RDS
- AWS Aurora
NoSQL
- AWS DynamoDB (JSON)
- AWS ElastiCache (Key-Value)
- Neptune (Graph)
- AWS DocumentDB (MongoDB)
- AWS Keyspaces (Cassandra)
Object Store
- S3
Data Warehouse
- AWS Redshift
- AWS Athena
- AWS EMR
Search
- AWS OpenSearch (free text, unstructured search)
Graphs
- AWS Neptune
Ledger
- AWS Quantum Ledger Database
- AWS Managed Blockchain
Time Series
- AWS Timestream

AWS RDS

Postgres, MySQL, Oracle, MSSQL, MariaDB
For Relational Databases (SQL, JOIN, Table, Column)
Additional security through IAM, Security Groups, SSL
Support for auto-scaling, Read Replicas, and multiple Availability Zones

High Availability

Can provision DB Instances in Primary/Standby or Read Replica/Standby within the same AWS Region
- If so configured, Standby will be promoted to the Primary DB Instance (say, of several Read Replicas).
- If so configured, Standby will be promoted to a Read Replica if the primary Read Replica fails.
- Provides failover support
- Synchronous data replication
DB Instances can be placed into Multi-AZ clusters.
- Read Replicas can be placed in differing Availability Zones within the same AWS Region.
- Read Replicas can be promoted to the Primary DB Instance.

Note that DB updates incur downtime.

RDS Proxy

Allows apps to pool and share DB connections established with a database
Handles failovers itself and reduces failover time by 66%
Enforces IAM authentication for your databases
Is never publicly accessible (must be accessed from VPC)

AWS Aurora

Compatible with MySQL and Postgres
Highly distributed
- Stored in 6 replicas
- Across 3 Availability Zones
Self-healing, high availability, auto-scaling

Aurora Global Databases

AWS Aurora Global Databases are singular database instances that span multiple AWS Regions (as opposed to AWS DynamoDB Global Tables which are comprised of many replicated tables treated as one).

AWS DynamoDB

Managed serverless NoSQL database
Provisioned and optional auto-scaling capacity
DAX cluster for read cache
Automated backups up to 35 Days
Event processing - DynamoDB Streams integrate with AWS Lambda or Kinesis Data Streams
Highly available, multiple Availability Zones
Decoupled Reads and Writes

DynamoDB Accelerator

DynamoDB Accelerator (DAX) is a fully managed in-memory cache for AWS DynamoDB offering 10x performance.
Deployed as a cluster.

DynamoDB Global Tables

AWS DynamoDB Global Tables are comprised of many replicated tables distributed across several AWS Regions so that they:

Are treated as one sharing
Share the same primary key schema

AWS ElastiCache

Caches database data using Redis or Memcached:

Redis:
- Supports Sets and Sorted Sets
- Backup and restore features
- Read replicas for High Availability
- Multiple Availability Zones
Memcached:
- No High Availability
- No backup and restore
- Multithreaded

AWS Neptune

Fully managed Graph Database
Highly available across 3 Availability Zones
Up to 15 read replicas

AWS Keyspaces

A managed Apache Cassandra-compatible database service
Tables are replicated 3 times across multiple Availability Zones
Auto-scales Tables up and down based on traffic
Uses Cassandra Query Language (CQL)

AWS Quantum Ledger

2-3x better performance than common ledger blockchain frameworks
Can use SQL
Fully managed, serverless, with high availability replication across 3 Availability Zones
An immutable ledger

AWS SAA-C03: S3

Replication
(File) Versioning

AWS S3 Storage Classes

S3 Standard
- S3 Standard
S3 Intelligent-Tiering
- S3 Intelligent-Tiering
S3 Standard-Infrequent Access
- S3 Standard-IA
S3 One Zone-Infrequent Access
- S3 One Zone-IA
Archive
- S3 Glacier Instant Retrieval
- S3 Glacier Flexible Retrieval
- S3 Glacier Deep Archive

Note S3 Glacier has been renamed S3 Glacier Flexible Retrieval.

Pricing

AWS users pay for:

Hosting data in AWS S3
Updating or Copying data already in AWS S3
Requests made against items hosted in AWS S3

AWS users don't pay for:

There is no cost for uploading data into AWS S3 itself
- Although one might pay for transmitting data into a VPC or across AWS Regions

AWS S3 Data Retention

Glacial Vaults
S3 Object Lock - Retention Mode
- Governance mode - some special permissions can alter
- Compliance mode - no one one can alter
S3 Object Lock - Retention Period
- Legal Hold - locked until removed
- Retention Period - a specified period of time

AWS S3 Bucket Security Features

MFA -Multi-Factor Authentication
- Can be required for deletes
- Used to protect resources
By URL:
- CORS - Cross-Origin Resource Sharing - restrict resource access when not on same Domain
- Pre-Signed URLs - white list which URLs S3 GET / PUT requests can come from
File Encryption - Server-Side Encryption (SSE)
- SSE-S3 - default
- SSE-KMS - SSE with AWS KMS
- SSE-C - SSE with Customer Provided Keys
Bucket Policies

Other Features

S3 Batch Operations - use S3 Batch and the S3 API
Supports multi-part uploading
S3 Transfer Accerlation uses intelligent routing to reduce the time and distance it takes to upload and download files from AWS S3
Versioning
Supports static site hosting
S3 Origins specify where AWS S3 gets content from to serve to viewers.
- Examples:
  - An S3 Bucket
  - An HTTP server running on AWS EC2

Static Websites

Allowed URL formats:

http://bucket-name.s3-website.Region.amazonaws.com
http://bucket-name.s3-website-Region.amazonaws.com

Resources and Links

https://aws.amazon.com/s3/storage-classes/

AWS SAA-C03: EC2

AWS EC2 Instance Purchasing Options

On-Demand - Pay by the second
Reserved - Reserve for 1 or 3 years
Spot - Request unused EC2 instances on the fly
Dedicated Hosts - Pay for a physical host that is fully dedicated to running your instances
Dedicated Instances - Pay by the hour for instances that run on single-tenant hardware
On-Demand Capacity Reservations - Reserve capacity for your instances in a specific Availability Zone for any duration.

Scheduled Reserved instances aren't presently offered.

Savings Plans can be used to reduce costs by making a commitment to a consistent amount of usage for 1 or 3 years.

AWS EC2 Reserved Instances

Reserved Instances - Reserved for 1 or 3 years.

Generally speaking, All Upfront payments will be lower in the long-run than No Upfront payments.

All Upfront - Complete payment at the start regardless of hours eventually used
Partial Upfront - Portion paid at the start with the remainder being billed at a fixed rate regardless of hours eventually used
No Upfront - Billed at a fixed rate regardless of hours eventually used

Reserved Instances have a Convertible payment option:

Convertible - Can be exchanged with another Convertible Reserved Instance
You cannot exchange:
- All Upfront Reserved Instances for No Upfront Convertible Reserved Instances.
- Partial Upfront Reserved Instances for No Upfront Convertible Reserved Instances.

AWS EC2 Instance Types

General Purpose - Balanced storage, compute, and memory
- M nomenclature (M5a, M6zn, etc.)
Compute Optimized - High-performance computing
- C nomenclature (C6g, C6i, etc.)
Memory Optimized - Large-in-memory instances. Great high-memory use resources like MySQL, Caching
- R nomenclature (R6a, R5n, etc.)
Storage Optimized - High, sequential read and write access to very large data sets on local storage. Great for databases
- D, I, H nomenclature (H1, D3, I3 etc.)

Public vs. Private IP

Public IP - Used on the public, global, internet/web. Not too Public IP Addresses can be the same
Private IP - Used within private, subnets

Placement Groups

Cluster - Same rack, Availability Zone. Fastest but most susceptible to risk factors
Spread - All the EC2 Instances are deployed on different hardware, Availability Zones, etc.
- Maximizes High Availability
- Limited to 7 Instances per Availability Zone
Partitions - Think muliple Cluster Placement Groups spread across multiple Availability Zones
- Up to 100s of Instances per Partition
- Up to 7 Partitions per Availability Zone

Elastic Network Interfaces

Can be attached and detached to EC2 Instances within the same Availability Zone
Used to assign a fixed Public or Private IP Address

AWS EC2 Hibernate

Stores RAM into a persistent state on an encrypted root EBS volume
Relaunching or restarting the Instance is much faster

Instance Store, EBS, and EFS

Root Volumes:

Can be an Instance Store
- Limited to 10GB
- Ephemeral stores for use with temporary data
Or a EBS Backed Root Volume
- Limited to 1TB

Comparison:

Instance Store:
- 1-1 with an Instance
- Has good I/O performance since they are directly attached
- They're ephemeral however and when they're lost, the Instance is lost and when they are turned off all persisted data is lost
Elastic Block Storage:
- Attach to one Instance at a time
- Locked at the Availability Zone (cannot be moved to another Availability Zone without a Restoration Snapshot)
- Better for long-term storage than Instance Stores
Elastic File Storage:
- Attaches to multiple (hundreds) Instances across the same Availability Zone a time
- Not limited to a single Availability Zone however
- Typically more expensive
- Networked storage

Load Balancers

AWS offers Elastic Load Balancer as a managed service. It comes in a few varieties:

Application Load Balancer:
- Layer 7 (HTTP), HTTP/2, WebSocket, HTTPS
- Application balancing
- Routing based on URL Path, Hostname, Query, Headers
- Routes to EC2 Instances, ECS Tasks, Lambda Functions, IP Address
Network Load Balancer:
- Layer 4 (TCP), TCP/UDP forwarding
- Extreme performance
- Routes based on IP Address not specific AWS service
Gateway Load Balancer:
- Layer 3 (Network), IP packets
- Uses Route Tables to route traffic for an entire VPC
- Primary use is to be the single place for all inbound traffic: firewall, security monitoring, packet analysis, etc.

Classic Load Balancers are being deprecated at the end of 2022.

The Sticky Session feature ensures that users are only connected to the same EC2 Instance (and the same application session, context).

Cross Zone Load Balancing:

Load balancing is split between all Instances across all Availability Zones
Otherwise each Instance in an Availability Zone will divide the assigned load balancing weight (for that Availability Zone) by the total number of Instances within that single Availability Zone

Auto-Scaling Groups

EC2 Instances can be combined into Auto-Scaling Groups
- EC2 Auto Scaling Launch Templates
- EC2 Auto Scaling Launch Configuration
- As a general rule of thumb: EC2 Auto Scaling Launch Templates > Launch Configurations
  1. They (Templates) provide moer configuration features
  2. They support multiple versions
  3. Templates are AWS-recommended
  4. The EC2 Instance with the oldest Launch Configuration is terminated first
They create new Instances and terminate them based on configureable triggers and Dynamic Scaling Policies
- For example: CloudWatch alarms
Auto-Scaling Group Minimum and Maximum Capacities apply to the total number of EC2 Instances across all Availability Zones

Lifecycle

Scaling Out takes precedence to Scaling In
- New EC2 Instances are launched before new ones are removed
Scaling In terminates EC2 Instances
- By default, the EC2 Instance with the oldest Launch Configuration is terminated first

Resources and Links

AWS SAA-C03: Containers

AWS Elastic Container Service

AWS ECS automatically increases and decreases the number of ECS Tasks.

ECS Auto Scaling Uses AWS Application Auto Scaling.
Target Tracking - scale based on a target CloudWatch Metric value.
Step Scaling - scale based on specified CloudWatch Alarm.
Scheduled Scaling - scale based on date/time.
If using the EC2 Launch Type:
- Auto Scaling Group Scaling - based on CPU usage.
- ECS Cluster Capacity Provider - allows EC2 Instances to be added based on missing capacity.

ECS Launch Types:

EC2
- Launch Docker containers on AWS.
- User provisions and mainttains the infrastructure (underlying EC2 Instances).
Fargate
- User just creates the Task Definitions.
- Serverless computing.
- User doesn't manage the underlying EC2 Instances.

IAM Roles:

EC2 Instance Profile
- EC2 Launch Type only.
ECS Task Role
- Assigned to a Task.

AWS Elastic Container Registry

Store and manage Docker images on AWS.
Private and Public repository.
Backed by S3.
Access via IAM permission.

AWS Elastic Kubernetes Service

Supports ECS.
Managed Node Groups.
Self-Managed Nodes.
AWS native solution for Kubernetes.

AWS AppRunner

Fully managed app service.
Builds and deploys apps.

AWS SAA-C03: Serverless

Serverless Computing is a paradigm where infrastructure is sold as a service (IAAS) in a fully managed way (abstracting away the underlying bare metal and operating system resources).

AWS Fargate

The user creates Task Definitions but AWS manages the rest of the ECS infrastructure
Limited to:
- 100 Tasks per Region per Account (default)
- 1,000 Tasks per Service
- By Amazon ECS Service Quotas (limits)

Consult the Elastic Container Service article.

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-quotas.html

AWS Lambda

A user creates Lambda Functions but doesn't have to manage the underlying infrastructure to execute them.
Lambda Functions are associated with specific Endpoints in AWS API Gateway and are invoked using standard HTTP REST methods and URL context paths.
Lambda Functions are stateless.
Indeed, they are ideal for stateless
There's a small delay when a Lambda Function is first called.
- A Lambda Function Context is created from a Cold state (the underlying resources are initialized and made available).
- However, a Lambda Function Contexts exists for 15 minutes in a Hot state.
- So, sequential calls will execute without the initial delay.
Lambda Functions will timeout after 15 minutes.
The default maximum number of simultaneous concurrent connections for a single Lambda Function is 1000 within the same AWS Region (this can be increased by request).

IAM Policies

Execution Roles - grant a Lambda Function permission to access other resources or perform certain operations.
Resource-Based Policy - how a Lamba Function itself can be used, invoked, or called by users or other services.

AWS API Gateway

Connect AWS Lambda Functions to API Gateway Endpoints.
Associate each endpoint with HTTP methods (PUT, POST, GET, DELETE, PATCH, OPTIONS).
Can define HTTP Request and Response Schemas.

AWS Step Functions

For sequential or "chained" operations that might require a lengthy or significant amount of execution time.

Serverless Stack

A commonly found and fully Serverless stack will comprise:

AWS DynamoDB - fully managed serverless DB.
AWS DynamoDB DAX - for Caching and read acceleration
AWS Lambda
AWS Cognito - for identity management and user authentication

AWS Proton

AWS Proton standardizes serveless architecture deployments.

Resources and Links

AWS SAA-C03: Machine Learning

Image Recognition

AWS Rekognition

Facial analysis and search using Machine Learning (ML) for user verification.
Find objects, people, text, and images in photos or video.

Speech and Text

AWS Transcribe

Automatically convert speech to text.
Deep Learning (Automatic Speech Recognition - ASR)

AWS Polly

Convert text into speech.

AWS Translate

Language translation.

AWS Lex + Connect

Automatic Speech Recognition (ASR) to convert speech into text.
Natural Language Understanding to recognize the intent of text, callers.
For chatbots, call center bots.
Receive calls, create contact flows, cloud-based virtual contact center.

AWS Comprehend

Natural Language Processing (NLP) to find insights and relationships in text.
Fully managed, serverless.
Specialized service for unstructured medical/clinical text (HIPAA).

AWS Textract

Extract text, handwriting, and data from any scanned documents.
Extrac data from forms and tables.

Fully Managed Services

AWS SageMaker

Fully managed service for developers to build Machine Learning models.

AWS Forecast

Fully managed service for developers to build highly accurate forecasts.

AWS Kendra

Fully managed document search service.
Extracts answers from within a document.
Natural language search capabilities.

AWS Personalize

Fully managed service for making real-time personalized recommendations.

COMPTIA SY0-701: Overview

The COMPTIA Security+ SY0-701 exam divides into five general security topics:

General Security Concepts
Threats, Vulnerabilities, and Mitigations
Security Architecture
Security Operations
Security Program Management and Oversight

Resources and Links

COMPTIA SY0-701: General Security Concepts

Security Controls

Compare and contrast various types of security controls:

Categories
- Technical
  - Also, called logical security controls that are implemented with technology and executed by computer systems.
  - "Hardware and software controls used to provide automated protection to the IT system or applications." (CNSS 4009)
  - Examples: IDs, Firewalls, Encryption
- Managerial
  - Also, called administrative security controls that are documented policies and prioritize risk reduction.
  - Examples: Organizational security policies (the procedures not the implementation in say AWS IAM), risk assessments, and security awareness training.
- Operational
  - The security controls (i.e., safeguards or countermeasures) for an information system that're primarily implemented and executed by people (as opposed to systems).
  - Security controls that keep machines and devices running correctly.
- Physical
  - Set of measures taken to protect business assets (including information, hardward, people, and data) from physical threats.
Control Types
- Preventive
  - Prevents incidents
  - Policies, standards, orocedures, firewalls, configurations, etc.
- Deterrent
  - Deters would-be attackers
  - Examples: Locks, vestibules, lighting, signs, steel doors, etc.
- Detective
  - Identifies problems once they have occurred
  - Examples: audits, investigations, anomoly reports, etc.
- Corrective
  - Fixes a problem after it's occurred
  - Examples: patch, fix, new policy, disciplinary action, etc.
- Compensating
  - Alternative taken to address a security concern
  - Examples: reverting to a prior secure version instead of patching due to lack of expertise or time, using automated detection systems in light of low staff count, etc.
- Directive
  - Established to bring about desired outcomes
  - Examples: a new guidance that aims to make it easier to apply security fixes in the future, etc.

Security Concepts

Summarize fundamental security concepts:

Confidentiality, Integrity, and Availability (CIA)
Non-repudiation
- Cannot dispute authorship or origin of something. (Cannot repudiate something.)
Authentication, Authorization, and Accounting (AAA)
- Authenticating People
- Authenticating systems
- Authorization models
Gap Analysis
Zero Trust
- Control Plane
  - Responsible for defining and managing security policies.
  - Adaptive Identity
  - Threat Scope Reduction
  - Policy-Driven Access Control
  - Policy Administrator (PA)
  - Policy Engine (PE)
- Data Plane
  - Responsible for enforcing and executing the security policies defined by the Control Plane.
  - Implicit Trust Zones
  - Subject/System
  - Policy Enforcement Point (PEP)
- Refer to: https://konghq.com/learning-center/cloud-connectivity/control-plane-vs-data-plane
Physical Security
- Bollards
  - Vertical posts that create a physical barrier outside a building.
- Access Control Vestibule
  - A secured entrance booth that someone must pass through to gain entry to a building.
- Fencing
- Video Surveillance
- Security Guard
- Access Badge
- Lighting
- Sensors
  - Infrared
  - Pressure
  - Microwave
  - Ultrasonic
Deception and Disruption Technology
- Honeypot
  - A a resource that's designed to detect, deflect, or distract some unauthorized use of a system.
  - Legitimate users should not access or use the Honeypot.
  - Example: a subsystem that secretly tracks unauthorized IP Addresses by posing as a security vulnerability
- Honeynet
  - A Honeypot that's an entire network that's used to detect vulnerabilities.
  - Uses servers, routers, logging, switches, firewalls, and other devices to monitor and track would be hackers.
- Honeyfile
  - A Honeypot that's a file that tracks a would be hacker.
  - Legitimate users should not access or use the Honeyfile.
- Honeytoken
  - An identifier that's used to flag a would be attacker and their actions on a network.
  - Examples: a cookie or bearer token that's generated when a user accesses something in an unauthorized way and then is subsequently used to track the whereabouts and actions of that user.

Change Management

Explain the importance of change management processes and the impact to security:

Business Processes Impacting Security Operation
- Approval Process
- Ownership
- Stakeholders
- Impact Analysis
- Test Results
- Backout Plan
- Maintenance Window
- Standard Operating Procedure
Technical Implications
- Allow Lists/Deny Lists
- Restricted Activities
- Downtime
- Service Restart
- Application Restart
- Legacy Applications
- Dependencies
Documentation
- Updating Diagrams
- Updating Policies/Procedures
Version Control

Cryptographic Solutions

Explain the importance of using appropriate cryptographic solutions:

Public Key Infrastructure (PKI)
- Public Key
  - Shared and potentially visible publicly.
  - Example: an SSH Public Key added to GitHub.
- Private Key
  - Known only to a specific user and never (or shouldn't be) shared publicly.
  - Example: an SSH Private Key for connecting a terminal to GitHub that's stored locally in ~/.ssh.
- Key Escrow
  - An arrangement where keys are held in escrow so an authorized third-party can get access to them.
Encryption
- Level
  - Full-Disk Encryption (FDE)
  - Partition
  - File
  - Volume
  - Database
  - Record
Transport/Communication
Asymmetric
- Includes Public-key cryptography (PKI, above).
- A Key is divided into pairs with distinct levels of visibility between the two: Public and Private.
- RSA
Symmetric
- Where only one Key is used to encrypt or decrypt data.
- AES
Key Exchange
- Where two or more parties coordinate the sharing or exchange of a Key deliberately created for the exchange.
Algorithms
- Advanced Encryption Standard (AES)
  - Trusted by U.S. Government.
  - Typically, 128, 192, or 256 (in increasing order of encryption strength).
  - Widely thought to be resistant to Quantum tampering/decryption (although this has not been formally proven).
  - Symmetric and lower strengths are susceptible to Brute-Forcing attacks.
- Rivest–Shamir–Adleman (RSA)
  - Used in PGP and GPG.
  - Asymmetric Public and Private Key.
Key Length
Tools
- Trusted Platform Module (TPM)
  - A dedicated microcontroller designed to secure hardware through integrated cryptographic Keys.
- Hardware Security Module (HSM)
  - A physical computing device that protects and manages secrets.
- Key Management System
- Secure Enclave
  - The Secure Enclave is a dedicated secure subsystem integrated into Apple chips isolated from the main processor to provide an extra layer of security.
Obfuscation
- Steganography
  - Concealing a file, message, image, or video within another file, message, image, or video.
- Tokenization
- Data Masking
Hashing
- Any function or technique that maps some value to an entry or key.
Salting
- A value that's supplied along with an encryption algorithm (Cipher) and Plaintext value to introduce variance into the resultant encrypted text.
Digital Signatures
Key Stretching
Blockchain
Open Public Ledger
Certificates
- Certificate Authorities (CA)
  - Distinction between a CA and PKI:
    - A CA issues Certificates.
    - PKI is a system of digital Certificates, Public Keys, and Private Keys that are used to secure communications.
- Certificate Revocation Lists (CRLs)
- Online Certificate Status Protocol (OCSP)
- Self-Signed
- Third-Party
- Root of Trust
- Certificate Signing Request (CSR) Generation
- Wildcard

Resources and Links

COMPTIA SY0-701: Threats, Vulnerabilities, and Mitigations

Threat Actors

Compare and contrast common threat actors and motivations:

Threat Actors
- Nation-State
- Unskilled Attacker
- Hacktivist
- Insider Threat
- Organized Crime
- Shadow IT
Attributes of Actors
- Internal/External
- Resources/Funding
- Level of Sophistication/Capability
Motivations
- Data Exfiltration
- Espionage
- Service Disruption
- Blackmail
- Financial Gain
- Philosophical/Political Beliefs
- Ethical
- Revenge
- Disruption/Chaos
- War

Attack Surfaces

Explain common threat vectors and attack surfaces:

Message-Based
- Email
- Short Message Service (SMS)
- Instant Messaging (IM)
Image-Based
File-Based
Voice Call
Removable Device
Vulnerable Software
- Client-Based vs. Agentless
Unsupported Systems and Applications
Unsecure Networks
- Wireless
- Wired
- Bluetooth
Open Service Ports
Default Credentials
Supply Chain
- Managed Service Providers (MSPs)
- Vendors
- Suppliers
Human Vectors/Social Engineering
- Phishing
  - Obtaining sensitive data through email and misleading URLs/links.
- Vishing
  - Obtaining sensitive data through video and/or telephony services.
  - Think video-based Phising.
- Smishing
  - Obtaining sensitive data through SMS text messages/messaging services.
  - Think, SMS text-message Phishing.
- Misinformation/Disinformation
- Impersonation
- Business Email Compromise
- Pretexting
  - Providing a false/fake reason to trick someone into divulging sensitive data.
- Watering Hole
  - Identifying a popular or commonly used website as a likely resource that an organization uses to infect it with malware.
  - Think: infecting the "gathering place" where many users within an organization are likely to digitally congregate.
- Brand Impersonation
- Typosquatting
  - Buying a Domain Name that's similar to a popular and well-trafficked website to trick people who accidentally visit it (when they enter typos or through misspellings by mistake).

Vulnerabilities

Explain various types of vulnerabilities:

Application
- Memory Injection
- Buffer Overflow
- Race Conditions
  - Time-of-Check (TOC)
  - Time-of-Use (TOU)
- Malicious Update
Operating System (OS)-Based
Web-Based
- Structured Query Language Injection (SQLi)
  - When input text is used to execute unwanted and unexpected SQL queries/commands to modify a database or return sensitive data.
- Cross-Site Scripting (XSS)
  - An exploit where malicious code from a compromised (or attacking) website is used to execute malicious code on another.
- These typically involve a lack of input validation and sanitization (escaping characters/text, preventing input text from being used in queries or executed code, etc.).
Hardware
- Firmware
- End-of-Life
- Legacy
Virtualization
- Virtual Machine (VM) Escape
- Resource Reuse
Cloud-Specific
Supply Chain
- Service Provider
- Hardware Provider
- Software Provider
Cryptographic
Misconfiguration
Mobile Device
- Side Loading
- Jailbreaking
Zero-Day
- An attack that occurs before software developers can find a fix.
- Refers to the idea that software developers have "zero days to find a fix" since it's already being exploited.

Indicators of Malicious Activity

Given a scenario, analyze indicators of malicious activity:

Malware attacks
- Ransomware
  - Blocks access to a resource or data until some ransom is paid.
- Trojan
  - Malware that's disguised as a beneficial or standard program.
  - Misleads a user as to its true intent.
- Worm
  - Replicates itself to spread to other computers on a network.
  - Stand-alone malicious programs that don't require user interaction (as opposed to a Virus).
- Spyware
  - Malware that collects and transmits data about a user, computer, or device.
- Bloatware
  - Applications or programs that are unwanted.
  - Often installed along with some desired app.
- Virus
  - Can copy itself to other computers and resources on a network and infect a computer without the knowledge or permission of a user.
  - Requires some user-initiated interaction (e.g. - downloading and opening an unknown file).
- Keylogger
  - Records a user's keystrokes and transmits them to obtain passwords or other sensitive information.
- Logic Bomb
  - A type of malicious code embedded in software that is activated/triggered by some specific condition.
- Rootkit
  - Used to obtain administrator-level control over a device, computer, or network.
Physical Attacks
- Brute Force
- Radio Frequency Identification (RFID) Cloning
- Environmental
Network Attacks
- Distributed Denial-of-Service (DDoS)
  - Amplified
    - Uses multiple DNS Resolvers that are configured to accept recursive DNS Queries to overwhelm a victim with traffic.
    - DNS request-response traffic is made to originate from the victim resource and is then directed back to the victim overwhelming it with traffic.
  - Reflected
    - A kind of DDoS attack where the attacker forges the source address of request packets from a DNS Resolver overwhelming the victim resource with traffic generated from the request responses.
- Domain Name System (DNS) Attacks
  - A category of attacks that exploit the DNS infrastructure or protocol.
  - An attacker modifies the DNS Query and sends bogus responses before a legitimate response arrives.
  - Examples:
    - Amplified and Reflected DDoS attacks (above).
    - DNS Hijacking involves an attacker modifying a DNS Server and/or A Record to point a Domain Name to another IP Address.
  - Refer to: https://bluecatnetworks.com/blog/four-major-dns-attack-types-and-how-to-mitigate-them/
- Wireless
- On-Path
  - Also called a Man-in-the-Middle (MIM) attack.
  - An attacker interposes oneself between two or more parties without authorization or their knowledge.
- Credential Replay
- Malicious Code
Application Attacks
- Injection
  - Exploits text or input serialization vulnerabilities to execute unwanted, unexpected, or untrusted code.
- Buffer Overflow
  - An exploit where the amount of data exceeds the available storage or memory.
- Replay
  - An exploit where transmitted authentication or access control information is recorded and then reused/replayed for unauthorized access or effect.
- Privilege Escalation
  - An exploit where permissions, access, or privileges are increased to gain unauthorized access, visibility, or privilege.
- Forgery
  - An exploit involving the use of credentials to gain unwanted and unauthorized permission to carry out some task.
- Directory Traversal
  - Where input text is exploited to traverse directories, read or find files within the container, or gain unauthorized access to resources.
Cryptographic Attacks
- Downgrade
  - An attack that forces a network channel to switch or change to an unprotected and less secure data transmission format.
- Collision
  - An attack that attempts to find two input Strings that generate the same Hash (Hash Collision) to determine the hashing method.
- Birthday
  - A kind of Brute Force attack that exploits the likelihood of certain seemingly low-probability events to determine the method of hashing (by Hash Collision).
Password attacks
- Spraying
  - A specific kind of Brute Force attack where the same passwords are tried against multiple accounts.
- Brute Force
  - An attack that uses trial and error to crack a password, credential, or secret, or to obtain unauthorized access to a resource, network, device, or computer.
Indicators
- Account Lockout
- Concurrent Session Usage
- Blocked Content
- Impossible Travel
- Resource Consumption
- Resource Inaccessibility
- Out-of-Cycle Logging
- Published/Documented
- Missing Logs

Mitigation Techniques

Explain the purpose of mitigation techniques used to secure the enterprise:

Segmentation
Access Control
- Access Control List (ACL)
- Permissions
Application Allow List
Isolation
Patching
Encryption
Monitoring
Least Privilege
Configuration Enforcement
Decommissioning
Hardening Techniques
- Encryption
- Installation of Endpoint Protection
- Host-Based Firewall
- Host-Based Intrusion Prevention System (HIPS)
- Disabling Ports/Protocols
- Default Password Changes
- Removal of Unnecessary Software

Resources and Links

https://bluecatnetworks.com/blog/four-major-dns-attack-types-and-how-to-mitigate-them/

COMPTIA SY0-701: Security Architecture

Architecture Models

Compare and contrast security implications of different architecture models:

Architecture and Infrastructure Concepts
- Cloud
  - Responsibility Matrix
  - Hybrid Considerations
  - Third-Party Vendors
- Infrastructure as Code (IaC)
- Serverless
- Microservices
- Network Infrastructure
  - Physical Isolation
    - Air-Gapped
- Logical Segmentation
- Software-Defined Networking (SDN)
- On-Premises
- Centralized vs. Decentralized
- Containerization
- Virtualization
- IoT
- Industrial Control Systems (ICS)/Supervisory Control and Data Acquisition (SCADA)
- Real-Time Operating System (RTOS)
- Embedded Systems
- High Availability
Considerations
- Availability
- Resilience
- Cost
- Responsiveness
- Scalability
- Ease of Deployment
- Risk Transference
- Ease of Recovery
- Patch Availability
- Inability to Patch
- Power
- Compute

Security Principles

Given a scenario, apply security principles to secure enterprise infrastructure:

Infrastructure Considerations
- Device Placement
- Security Zones
- Attack Surface
- Connectivity
- Failure Modes
  - Fail-Open
    - Where a failure results in the "gates opening" (the resource or service allows access).
  - Fail-Closed
    - Where a failure results in the "gates closing" (the resource or service disallows or severs access).
- Device Attribute
  - Active vs. Passive
  - Inline vs. Tap/Monitor
- Network Appliances
  - Jump Server
  - Proxy Server
  - Intrusion Prevention System (IPS)/Intrusion Detection System (IDS)
  - Load Balancer
  - Sensors
- Port Security
  - 802.1X
  - Extensible Authentication Protocol (EAP)
- Firewall Types
  - Web Application Firewall (WAF)
  - Unified Threat Management (UTM)
  - Next-Generation Firewall (NGFW)
  - Layer 4/Layer 7
    - Layer 4 (TCP), TCP/UDP forwarding
    - Layer 7 (HTTP), HTTP/2, WebSocket, HTTPS
Secure Communication/Access
- Virtual Private Network (VPN)
- Remote Access
- Tunneling
  - Transport Layer Security (TLS)
  - Internet Protocol Security (IPSec)
- Software-Defined Wide Area Network (SD-WAN)
- Secure Access Service Edge (SASE)
Selection of Effective Controls

Data Protection Concepts

Compare and contrast concepts and strategies to protect data:

Data Types
- Regulated
- Trade Secret
- Intellectual Property
- Legal Information
- Financial Information
- Human- and Non-Human-Readable
Data Classifications
- Sensitive
- Confidential
- Public
- Restricted
- Private
- Critical
General Data Considerations
- Data States
  - Data at Rest
  - Data in Transit
  - Data in Use
- Data Sovereignty
- Geolocation
Methods to Secure Data
- Geographic Restrictions
- Encryption
- Hashing
- Masking
- Tokenization
- Obfuscation
- Segmentation
- Permission Restrictions

Resilience and Recovery

Explain the importance of resilience and recovery in security architecture:

High Availability
- Load Balancing vs. Clustering
Site Considerations
- Hot
- Cold
- Warm
- Geographic Dispersion
Platform Diversity
Multi-Cloud Systems
Continuity of Operations
Capacity Planning
- People
- Technology
- Infrastructure
Testing
- Tabletop Exercises
- Fail Over
- Simulation
- Parallel Processing
Backups
- Onsite/Offsite
- Frequency
- Encryption
- Snapshots
- Recovery
- Replication
- Journaling
Power
- Generators
- Uninterruple Power Supply (UPS)

COMPTIA SY0-701: Security Operations

Common Security Techniques

Given a scenario, apply common security techniques to computing resources:

Secure Baselines
- Establish
- Deploy
- Maintain
Hardening Targets
- Mobile Devices
- Workstations
- Switches
- Routers
- Cloud Infrastructure
- Servers
- ICS/SCADA
  - Industrial Control Systems (ICS) and Supervisory Control and Data Acquisition (SCADA)
  - Are used to monitor industrial infrastructure processes and systems.
- Embedded Systems
- RTOS
- IoT Devices
Wireless Devices
- Installation Considerations
  - Site Surveys
  - Heat Maps
Mobile Solutions
- Mobile Device Management (MDM)
- Deployment Models
  - Bring Your Own Device (BYOD)
    - Corporate-Owned, Personally Enabled (COPE)
    - Choose Your Own Device (CYOD)W
- Connection Methods
  - Cellular
  - Wi-Fi
  - Bluetooth
Wireless Security Settings
- Wi-Fi Protected Access 3 (WPA3)
- AAA/Remote Authentication Dial-In User Service (RADIUS)
  - RADIUS uses UDP (as opposed to TACSYS+ which uses TCP).
- Cryptographic Protocols
- Authentication Protocols
Application Security
- Input Validation
- Secure Cookies
- Static Code Analysis
- Code Signing
Sandboxing
Monitoring

Security Implications of Proper Asset Management

Explain the security implications of proper hardware, software, and data asset management:

Acquisition/Procurement Process
Assignment/Accounting
- Ownership
- Classification
Monitoring/Asset Tracking
- Inventory
- Enumeration
Disposal/Decommissioning
- Sanitization
- Destruction
- Certification
- Data Retention

Vulnerability Management

Explain various activities associated with vulnerability management:

Identification Methods
- Vulnerability Scan
- Application Security
  - Static Analysis
  - Dynamic Analysis
  - Package Monitoring
- Threat Feed
  - Open-Source Intelligence (OSINT)
  - Proprietary/Third-Party
  - Information-Sharing Organization
  - Dark Web
- Penetration Testing
- Responsible Disclosure Program
  - Bug Bounty Program
- System/process Audit
Analysis
- Confirmation
  - False Positive
  - False Negative
- Prioritize
- Common Vulnerability Scoring System (CVSS)
- Common Vulnerability Enumeration (CVE)
- Vulnerability Classification
- Exposure Factor
- Environmental Variables
- Industry/Organizational Impact
- Risk Tolerance
Vulnerability Response and Remediation
- Patching
- Insurance
- Segmentation
- Compensating Controls
- Exceptions and Exemptions
Validation of Remediation
- Rescanning
- Audit
- erification
Reporting

Monitoring Concepts and Tools

Explain security alerting and monitoring concepts and tools:

Monitoring Computing Resources
- Systems
- Applications
- Infrastructure
Activities
- Log Aggregation
- Alerting
- Scanning
- Reporting
- Archiving
- Alert Response and Remediation/Validation
  - Quarantine
  - Alert Tuning
Tools
- Security Content Automation Protocol (SCAP)
- Benchmarks
- Agents/Agentless
- Security Information and Event Management (SIEM)
  - Combines log management, event correlation, and real-time monitoring.
- Antivirus
- Data Loss Prevention (DLP)
- Simple Network Management Protocol (SNMP) Traps
- NetFlow
- Vulnerability Scanners

Modify Security

Given a scenario, modify enterprise capabilities to enhance security:

Firewall
- Rules
- Access Lists
- Ports/Protocols
- Screened Subnets
IDS/IPS (Intrusion Detection System/Intrusion Prevention System)
- Trends
- Signatures
  - Refers to non-user-behavior patterns to identify potential security threats (e.g. - DDoS attacks, Directory Traversal attacks, etc.).
Web Filter
- Agent-Based
- Centralized Proxy
- Universal Resource Locator (URL) Scanning
- Content Categorization
- Block Rules
- Reputation
Operating System Security
- Group Policy
- SELinux
Implementation of Secure Protocols
- Protocol Selection
- Port Selection
- Transport Method
DNS Filtering
Email Security
- Domain-based Message Authentication Reporting and Conformance (DMARC)
  - Uses both DKIM and SPF to verify the authenticity of and sign emails.
  - Allows DMARC Policies to be specified to define how unauthenticated emails are handled.
  - This is the recommended way to implement email authentication and cryptographic signing.
  - Should be fewer incorrect email rejections, blocked emails, or emails counted as spam.
- DomainKeys Identified Mail (DKIM)
  - Signs emails to verify authenticate origin.
- Sender Policy Framework (SPF)
  - List of all Servers a Domain will send emails from.
  - Used to determine if an email is spam.
- Gateway
File Integrity Monitoring
DLP
Network Access Control (NAC)
Endpoint Detection and Response (EDR)/Extended Detection and Response (XDR)
- Extended Detection and Response (XDR)
  - Collects security information from multiple security layers and resources. (As opposed to only from Layer 4, Layer 7, or say a single Web Application and not its VPC traffic.)
User Behavior Analytics

Identity and Access Management

Given a scenario, implement and maintain identity and access management:

Provisioning/De-Provisioning User Accounts
Permission Assignments and Implications
Identity Proofing
Federation
Single Sign-On (SSO)
- Lightweight Directory Access Protocol (LDAP)
- Open Authorization (OAuth)
- Security Assertions Markup Language (SAML)
Interoperability
Attestation
Access Controls
- Mandatory
- Discretionary
- Role-Based
- Rule-Based
- Attribute-Based
- Time-of-Day Restrictions
- Least Privilege
Multifactor Authentication
- Implementations
  - Biometrics
  - Hard/soft Authentication Tokens
  - Security Keys
- Factors
  - Something You Know
  - Something You Have
  - Something You Are
  - Somewhere You Are
Password Concepts
- Password Best Practices
  - Length
  - Complexity
  - Reuse
  - Expiration
  - Age
- Password Managers
  - Passwordless
Privileged Access Management Tools
- Just-in-time Permissions
- Password Vaulting
- Ephemeral Credentials

Automation and Orchestration

Explain the importance of automation and orchestration related to secure operations:

Use Cases of Automation and Scripting
- User Provisioning
- Resource Provisioning
- Guard Rails
- Security Groups
- Ticket Creation
- Escalation
- Enabling/Disabling Services and Access
- Continuous Integration and Testing
  - Continuous Integration and Continuous Delivery (CI/CD)
    - Continuous Delivery stops short of automating Production deployments by requiring manual approval to do so.
  - Continuous Integration and Continuous Deployment (CI/CD)
    - Continuous Deployment deploys code all the way to Production automatically.
- Integrations and Application Programming Interfaces (APIs)
Benefits
- Efficiency/Time Saving
- Enforcing Baselines
- Standard Infrastructure Configurations
- Scaling in a Secure Manner
- Employee Retention
- Reaction Time
- Workforce Multiplier
Other Considerations
- Complexity
- Cost
- Single Point of Failure
- Technical Debt
- Ongoing Supportability

Incident Response

Explain appropriate incident response activities:

Process
- Preparation
- Detection
- Analysis
- Containment
- Eradication
- Recovery
- Lessons Learned
Training
Testing
- Tabletop Exercise
- Simulation
Root Cause Analysis
Threat Hunting
Digital Forensics
- Legal Hold
- Chain of Custody
- Acquisition
- Reporting
- Preservation
- E-discovery

Use Data Sources

Given a scenario, use data sources to support an investigation:

Log Data
- Firewall Logs
- Application Logs
- Endpoint Logs
- OS-Specific Security Logs
- IPS/IDS Logs
- Network Logs
- Metadata
Data Sources
- Vulnerability Scans
- Automated Reports
- Dashboards
- Packet Captures

Resources and Links

COMPTIA SY0-701: Security Program Management and Oversight

Security Governance

Summarize elements of effective security governance:

Guidelines
Policies
- Acceptable Use Policy (AUP)
- Information Security Policies
- Business Continuity
- Disaster Recovery
  - Disaster Recovery Plan (DRP)
- Incident Response
  - Incident Response Plan (IRP)
- Software Development Lifecycle (SDLC)
- Change Management
Standards
- Password
- Access Control
- Physical Security
- Encryption
Procedures
- Change Management
- Onboarding/Offboarding
- Playbooks
External Considerations
- Regulatory
- Legal
- Industry
- Local/Regional
- National
- Global
Monitoring and Revision
Types of Governance Structures
- Boards
- Committees
- Government Entities
- Centralized/Decentralized
Roles and Responsibilities for Systems and Data
- Owners
- Controllers
- Processors
- Custodians/Stewards

Risk Management Process

Explain elements of the risk management process:

Risk Identification
Risk Assessment
- Ad-Hoc
- Recurring
- One-Time
- Continuous
Risk Analysis
- Qualitative
- Quantitative
- Single Loss Expectancy (SLE)
  - Calculated cost of a single event.
- Annualized Loss Expectancy (ALE)
  - Calculated cost of all such events in a year.
- Annualized Rate of Occurrence (ARO)
  - Calculated frequency of events in a year.
- Probability
- Likelihood
- Exposure Factor
- Impact
Risk Register
- Key Risk Indicators
- Risk Owners
- Risk Threshold
Risk Tolerance
Risk Appetite
- Expansionary
- Conservative
- Neutral
Risk Management Strategies
- Transfer
- Accept
  - Exemption
  - Exception
- Avoid
- Mitigate
Risk Reporting
Business Impact Analysis
- Recovery Time Objective (RTO)
  - The maximum acceptable amount of time a service, system, or resource can be down before recovery is achieved.
  - Time-sensitive.
  - Example: A service can experience only 5 minutes of downtime to meet it's SLA.
- Recovery Point Objective (RPO)
  - The maximum acceptable amount of data (expressed as a time interval) that a service, system, or resource can be down before recovery is achieved.
  - Time-sensitive as understood through specified data timestamp.
  - Example: an RPO of 15 hours is set and the last valid data set had is from 10 hours ago. (No more than a 15 hour data gap can exist.)
- Mean Time to Repair (MTTR)
- Mean Time Between Failures (MTBF)

Third-Parties

Explain the processes associated with third-party risk assessment and management:

Vendor Assessment
- Penetration Testing
- Right-to-Audit Clause
- Evidence of Internal Audits
- Independent Assessments
- Supply Chain Analysis
Vendor Selection
- Due Diligence
- Conflict of Interest
Agreement Types
- Memorandum of Understanding (MOU)
  - States common areas of agreement but less binding than even a MOA.
  - No transfer of funds nor any enforcement of or obligation to.
- Memorandum of Agreement (MOA)
  - Conditional agreement but only slightly more binding than a MOU.
  - No transfer of funds nor any enforcement of or obligation to but often opening the door for future MSA or SOW's to be signed.
- Master Service Agreement (MSA)
  - Critical, foundational document, that covers business agreements.
  - High-level and binding.
- Work Order (WO)/Statement of Work (SOW)
  - Narrowly-scoped, very detailed, no room for ambiguity.
  - Usually subsequent agreements following up on MSA.
  - Often includes an SLA.
- Service-Level Agreement (SLA)
  - Exact conditions that must be met or that are expected.
  - Often included within an SOW.
- Non-Disclosure Agreement (NDA)
- Business Partners Agreement (BPA)
Vendor Monitoring
Questionnaires
Rules of Engagement

Effective Security Compliance

Summarize elements of effective security compliance:

Compliance Reporting
- Internal
- External
Consequences of Non-Compliance
- Fines
- Sanctions
- Reputational Damage
- Loss of License
- Contractual Impacts
Compliance Monitoring
- Due Diligence/Care
- Attestation and Acknowledgement
- Internal and External
- Automation
Privacy
- Legal Implications
  - Local/Regional
  - National
  - Global
- Data Subject
  - Any specific person that can be identified directly or indirectly by way of an identifier (name, ID number, account number, email address, or through certain attributes like age, cultural identity, and health characteristics.)
- Controller vs. Processor
  - A Data Controller determines the purposes and means of processing personal data.
  - A Data Processor processes personal data on behalf of the Data Controller.
- Ownership
  - Data Owners are senior leaders (executives) in charge of managing data and information relating to their specific areas of responsibility.
  - Data Custodians are responsible for ensuring the safety, storage, and maintenance of data assets (and no the data directly).
- Data Inventory and Retention
- Right to be Forgotten

Audits and Assessments

Explain types and purposes of audits and assessments:

Attestation
Internal
- Compliance
- Audit Committee
- Self-Assessments
External
- Regulatory
- Examinations
- Assessment
- Independent Third-Party Audit
Penetration Testing
- Physical
- Offensive
- Defensive
- Integrated
- Known Environment
- Partially Known Environment
- Unknown Environment
- Reconnaissance
  - Passive
  - Active

Security Awareness Practices

Given a scenario, implement security awareness practices:

Phishing
- Campaigns
- Recognizing a Phishing Attempt
- Responding to Reported Suspicious Messages
Anomalous Behavior Recognition
- Risky
- Unexpected
- Unintentional
User Guidance and Training
- Policy/Handbooks
- Situational Awareness
- Insider Threat
- Password Management
- Removable Media and Cables
- Social Engineering
- Operational Security
- Hybrid/Remote Work Environments
Reporting and Monitoring
- Initial
- Recurring
Development
Execution

Resources and Links

COMPTIA SY0-701: Miscellaneous Concepts

Windows Security

Encrypting File System (EFS)
- Encrypts specific files on any drive on a per-user basis.
Bitlocker
- Protects all personal and system files on the drive Windows is installed on.

Security Systems

Terminal Access Controller Access-Control System (TACACS+)
- Uses TCP (as opposed to RADIUS which uses UDP).
- Consult: https://www.cisco.com/c/en/us/support/docs/security-vpn/remote-authentication-dial-user-service-radius/13838-10.html

Kinds of Phishing

Spearphishing
- Highly personalized cyberattacks that target specific individuals or companies (typically involving Phishing).
- Whaling
  - Involves personalized Phishing attacks against VIP targets (CEO's, world leaders, executives, etc.).

Regulatory Designations

Personally Identifiable Information (PII)
- Regulated financial information
- Any identifying information about a customer including address, email, phone number, etc.
Protected Health Information (PHI)
- Any health-related or medical information about a customer that's covered by HIPAA.

Elliptic Curve Cryptography

Diffie-Hellman (DH)
- DH is not itself a Symmetric algorithm - it's an Asymmetric algorithm used to establish a shared secret for a Symmetric Key algorithm.
- Much less expensive to find a point on an elliptic curve than to Brute-Force prime numbers to find a solution.
- Elliptic Curve Cryptography (ECC)

RAID

RAID Configurations:

RAID 0
- Maximize read/write but without mirroring.
- Splits data up across multiple disks.
- Requires 2 disks at minimum.
RAID 1
- Automatically mirrors one disk on others.
- Requires 2 disks at minimum.
RAID 5
- Balances performance and redundancy.
- Requires 3 disks at minimum.
RAID 10
- Nests two RAID 1 sets within a RAID 0 configuration.
- Requires 4 disks at minimum.

Bluetooth

Bluejacking
- Is the practice of sending unsolicited messages over Bluetooth to Bluetooth-enabled devices without taking control of the device.

Important Acronyms

Simple Network Management Protocol (SNMP)
- Used to monitor and manage network devices.
- Devices share their state.

Wireless Security Protocols

WiFi encryption Standards:
- AES is the default Standard used in most contemporary WiFi security schemes.
- Temporal Key Integrity Protocol (TKIP)
  - Defined by the IEEE 802.11i Standard.
- Wired Equivalent Privacy (WEP)
  - Mostly deprecated.
By order of greatest security:
- Wi-Fi Protected Access (WPA2/3) with AES
  - These are often used in tandem with each other.
  - WPA implements the IEEE 802.11i Standard.
  - WPA typically uses the strongest encryption Standard (AES).
  - Occasionally, WPA will fall back to TKIP to support older devices.
    - This is increasingly deprecated and discouraged.
- Older versions of WPA (2 or 1).
- WPA with fallback to TKIP.
Deprecated:
- WEP
- TKIP

Key ISO Standards

ISO/IEC 27001
- Main Information Security Standard.
ISO/IEC 27018
- Main Standard specifying how to secure/protect Personally Identifiable Information (PII).
IEEE 802.11i
- Defines TKIP, WEP, and other Network/Wireless security mechanisms.

Resources and Links

Electrical Engineering: Watts Volts Amps Ohms

Basic Equivalences.

Resistance
- Expressed as Ohms (Ω) below.
- The aggregate opposition to Charge movement.
- The opposition to Electrons as they flow through material.
Current
- Expressed as Amps (A) below.
- Rate of flow of Charge.
Voltage
- Expressed as Volts (V) below.
- Difference in Electrical Potential.
Power
- Expressed as Watts (W) below.
- The amount of energy produced or consumed by an electrical device.

Electrons flow through material when there's an imbalance in Electrical Potential (Voltage). They flow at a certain rate (Current) which is modified by Resisting factors that inhibit how the Electrons move.

Ohms Law

Ω = V / A
Ω = V² / W
Ω = W / A²

Amps

A = V / Ω
A = W / V
A = √(W / Ω)

Volts

V = A * Ω
V = W / A
V = √(W * Ω)

Watts

W = V * A
W = V² / Ω
W = A² * Ω

Resources and Links

Electrical Engineering: Static Electricity

Electrons, Neutrons, and Protons

Electrons - are Negatively Charged Particles since they have a negative net Electrical Charge:
- 1.602176634 × 10−19 coulomb
Neutrons - are Neutrally Charged Particles since they have no net Electrical Charge.
Protons - are Positively Charged Particles they have a positive net Electrical Charge.

We usually count the Electric Charge of an Atom by the difference in Electrons and Protons.

Static Electricity

Static Electricity is an imbalance of Electrical Charge between two items (particularly on their surface).

Static Electricity shocks occur when there is:

An imbalance of Electrical Charge such that:
- One touching item is Negatively Charged (has many more Electrons)
- The other item is Positively Charged (has far fewer Electrons)

Static (Electricity) cling occur when there is:

When lightweight items stick to another owing to differences in Electrical Charge.
Example: confetti sticking to a plastic balloon.

Static (Electricity) hair raising occur when there is:

An excessive amount of positive Electrical Charge since two positively charged items will repel each other.
(Two positively charged items will repel each other. Two negatively charged items will repel each other.)

Static Electricity grounding:

Using a conductive material to keep a two items at the same, common, balanced, Electrical Charge.
To use a conductive cord, wire, or other material to prevent Static Electricity shocks.
- Commonly involves attaching metal wires to two items and connecting those wires to a ground block, wall, or metal object driven into the ground.

Prevention Techniques

Ground items in the manner described above.
Avoid certain materials like wool, polypropelene, etc. that have either a tendancy to generate Static Electricity or that lack the intrinsic ability to discharge it (e.g. - because they good insulators).
Touch small metal objects frequently and then touch those objects against some grounded item (a metal doorframe, a metal floor lamp, etc.).
Increase the amount of humidity in the air (since dry air increases the chance of Static Electricity).
- This option should be pursued only as a last resort around electronics since humidity can damage sensitive electrical components (water is conductive).