1 00:00:00,539 --> 00:00:03,539 foreign 2 00:00:10,500 --> 00:00:15,660 welcome everyone to the all things data 3 00:00:13,259 --> 00:00:18,960 track I'm super stoked for the lineup we 4 00:00:15,660 --> 00:00:20,640 have today my name is Ned I'm one of the 5 00:00:18,960 --> 00:00:22,199 hosts of The Specialist at the 6 00:00:20,640 --> 00:00:24,180 specialist track we also have Ray and 7 00:00:22,199 --> 00:00:25,619 Pedro who you'll see later on uh 8 00:00:24,180 --> 00:00:26,699 throughout the day they're down the 9 00:00:25,619 --> 00:00:29,160 front 10 00:00:26,699 --> 00:00:31,140 um the only minor housekeeping thing I'm 11 00:00:29,160 --> 00:00:33,300 gonna mention before we jump straight in 12 00:00:31,140 --> 00:00:35,640 is that we have a Discord Channel look 13 00:00:33,300 --> 00:00:38,460 for the data track uh Discord Channel 14 00:00:35,640 --> 00:00:39,840 pop questions in there for when folks 15 00:00:38,460 --> 00:00:41,879 are going to be taking questions if we 16 00:00:39,840 --> 00:00:44,340 have time and just say hello uh chat 17 00:00:41,879 --> 00:00:46,200 there's a big shout out to everyone in 18 00:00:44,340 --> 00:00:48,539 the online audience um it's great that 19 00:00:46,200 --> 00:00:53,219 you're tuning in as well so with that 20 00:00:48,539 --> 00:00:56,280 said let's jump into our proceedings 21 00:00:53,219 --> 00:00:58,739 and uh first up we have Christine seliga 22 00:00:56,280 --> 00:01:00,600 and long dang uh Dan they're going to be 23 00:00:58,739 --> 00:01:02,699 talking about uh unpacking the 24 00:01:00,600 --> 00:01:04,559 geospatial engineering toolbox an 25 00:01:02,699 --> 00:01:07,460 overview of data science techniques for 26 00:01:04,559 --> 00:01:10,140 spatial data just some quick intros 27 00:01:07,460 --> 00:01:11,820 Christine is the technical lead of wsp 28 00:01:10,140 --> 00:01:13,860 digital's data science and analytics 29 00:01:11,820 --> 00:01:14,939 team a scientist by training she has 30 00:01:13,860 --> 00:01:16,320 worked across a wide variety of 31 00:01:14,939 --> 00:01:17,939 different fields from computer science 32 00:01:16,320 --> 00:01:19,380 and software engineering to biomedical 33 00:01:17,939 --> 00:01:21,119 research fintech and now civil 34 00:01:19,380 --> 00:01:22,799 engineering she is continuously looking 35 00:01:21,119 --> 00:01:26,220 for new and interesting challenges to 36 00:01:22,799 --> 00:01:28,380 expand her Knowledge and Skills long is 37 00:01:26,220 --> 00:01:29,759 a junior data scientist WSB who likes 38 00:01:28,380 --> 00:01:32,220 all things math deep learning and 39 00:01:29,759 --> 00:01:34,500 computer science long is it most at home 40 00:01:32,220 --> 00:01:36,119 crunching through some algorithm coding 41 00:01:34,500 --> 00:01:38,759 problems with too much scratch paper and 42 00:01:36,119 --> 00:01:40,799 a pen when long is not at work you can 43 00:01:38,759 --> 00:01:42,240 find Long watching anime playing gacha 44 00:01:40,799 --> 00:01:43,860 games going on random walks if the 45 00:01:42,240 --> 00:01:45,960 weather is nice meeting friends and new 46 00:01:43,860 --> 00:01:49,140 people or trying to get a neural network 47 00:01:45,960 --> 00:01:52,579 loss to go down so please Round of 48 00:01:49,140 --> 00:01:52,579 Applause for our first talk 49 00:01:55,979 --> 00:01:58,619 yep 50 00:01:57,299 --> 00:01:59,939 um hi everyone 51 00:01:58,619 --> 00:02:02,700 um thank you for spending some of your 52 00:01:59,939 --> 00:02:04,860 time at the first half for this track 53 00:02:02,700 --> 00:02:06,119 um so I think when you think about it 54 00:02:04,860 --> 00:02:08,819 science you probably think about 55 00:02:06,119 --> 00:02:10,860 statistical techniques like the things 56 00:02:08,819 --> 00:02:13,020 that you can work on for tables like the 57 00:02:10,860 --> 00:02:14,640 things you find in Excel or you may be 58 00:02:13,020 --> 00:02:16,800 thinking about unstructured data like 59 00:02:14,640 --> 00:02:19,080 images and texts the things that you 60 00:02:16,800 --> 00:02:20,940 work with when using deep learning 61 00:02:19,080 --> 00:02:23,220 models like strategypt or Sable 62 00:02:20,940 --> 00:02:26,220 diffusion they're all cool and stuff but 63 00:02:23,220 --> 00:02:27,660 at my workplace wsv we have a lot of 64 00:02:26,220 --> 00:02:30,239 contact with civil engineering 65 00:02:27,660 --> 00:02:31,560 disciplines so one of the data type that 66 00:02:30,239 --> 00:02:33,840 we work a lot with is actually 67 00:02:31,560 --> 00:02:35,520 geospatial data these are everywhere 68 00:02:33,840 --> 00:02:38,220 they're very useful and in this 69 00:02:35,520 --> 00:02:40,379 presentation we would like to give you a 70 00:02:38,220 --> 00:02:42,720 brief overview of what geospatial data 71 00:02:40,379 --> 00:02:44,700 is what you can use it for we'll give 72 00:02:42,720 --> 00:02:47,459 you some overviews of patent libraries 73 00:02:44,700 --> 00:02:49,739 and techniques that we use and we'll 74 00:02:47,459 --> 00:02:51,900 give you some examples of the actual use 75 00:02:49,739 --> 00:02:54,060 cases or projects that we deliver to 76 00:02:51,900 --> 00:02:56,040 clients and give them insights from 77 00:02:54,060 --> 00:02:57,900 their geospatial data 78 00:02:56,040 --> 00:02:59,700 we'd like to begin by acknowledging that 79 00:02:57,900 --> 00:03:01,800 we're meeting on the traditional country 80 00:02:59,700 --> 00:03:03,420 of the Ghana people of the Adelaide 81 00:03:01,800 --> 00:03:05,940 pains and pay respects to the elders 82 00:03:03,420 --> 00:03:08,519 past presented emerging my name is long 83 00:03:05,940 --> 00:03:11,220 and second Sunday with me is Christine 84 00:03:08,519 --> 00:03:13,080 and we are data scientists at the team 85 00:03:11,220 --> 00:03:15,120 so the first question is probably the 86 00:03:13,080 --> 00:03:17,220 most important one is what is your 87 00:03:15,120 --> 00:03:17,879 spatial data and why is it important to 88 00:03:17,220 --> 00:03:19,620 us 89 00:03:17,879 --> 00:03:21,720 so the long and short story is that 90 00:03:19,620 --> 00:03:24,360 geospatial data represents all the 91 00:03:21,720 --> 00:03:26,099 physical things that surround us so it 92 00:03:24,360 --> 00:03:27,959 can represent the buildings this 93 00:03:26,099 --> 00:03:30,599 building this room so the chairs the 94 00:03:27,959 --> 00:03:32,040 roads around us and can represent the 95 00:03:30,599 --> 00:03:34,379 electrical lines that bring power into 96 00:03:32,040 --> 00:03:37,319 our homes the water pipes that bring 97 00:03:34,379 --> 00:03:39,659 water and waste away from our homes 98 00:03:37,319 --> 00:03:40,799 and Google Maps as you can see to the 99 00:03:39,659 --> 00:03:42,900 right it's probably one of the best 100 00:03:40,799 --> 00:03:45,180 example of how we use geospatial data in 101 00:03:42,900 --> 00:03:47,459 our everyday life so we use all this 102 00:03:45,180 --> 00:03:50,519 geospatial information about the layout 103 00:03:47,459 --> 00:03:52,799 of the buildings of the cities and the 104 00:03:50,519 --> 00:03:54,720 public transport networks to help me get 105 00:03:52,799 --> 00:03:57,060 to this convention center today 106 00:03:54,720 --> 00:03:58,799 we also use it to find places to go to 107 00:03:57,060 --> 00:04:00,420 on weekends for example 108 00:03:58,799 --> 00:04:02,280 and this is what geospatial data is 109 00:04:00,420 --> 00:04:03,780 important we want to use all of this 110 00:04:02,280 --> 00:04:05,760 important information to make better 111 00:04:03,780 --> 00:04:07,379 decisions not only how to move from 112 00:04:05,760 --> 00:04:09,599 place to place but also how to build 113 00:04:07,379 --> 00:04:12,000 better cities better environments and 114 00:04:09,599 --> 00:04:14,040 better communities so for example we may 115 00:04:12,000 --> 00:04:15,659 want to use your spatial data to find 116 00:04:14,040 --> 00:04:17,820 the best place to build new social 117 00:04:15,659 --> 00:04:20,400 housing new parks for just 118 00:04:17,820 --> 00:04:21,780 underprivileged communities we may want 119 00:04:20,400 --> 00:04:23,759 to understand how our infrastructure 120 00:04:21,780 --> 00:04:26,040 work how they are all connected where 121 00:04:23,759 --> 00:04:28,979 faults may happen and if they do happen 122 00:04:26,040 --> 00:04:31,020 where can we best solve them 123 00:04:28,979 --> 00:04:32,880 before we can delve into like solving 124 00:04:31,020 --> 00:04:34,139 all these important problems we first 125 00:04:32,880 --> 00:04:36,180 need to understand how we actually 126 00:04:34,139 --> 00:04:37,380 represent geospatial data for computers 127 00:04:36,180 --> 00:04:38,940 to understand 128 00:04:37,380 --> 00:04:40,979 there are many ways that you can 129 00:04:38,940 --> 00:04:42,660 represent your spatial data depending on 130 00:04:40,979 --> 00:04:44,820 whether you want to include verticality 131 00:04:42,660 --> 00:04:47,940 or not but for this presentation we'll 132 00:04:44,820 --> 00:04:49,979 mostly focus on the 2D case so you know 133 00:04:47,940 --> 00:04:52,199 like a map where things live in two 134 00:04:49,979 --> 00:04:54,419 dimensions in two Dimensions everything 135 00:04:52,199 --> 00:04:56,940 left on the coordinate system so they 136 00:04:54,419 --> 00:04:58,919 can be represented as Vector data and 137 00:04:56,940 --> 00:05:00,419 these are the intuitive things that you 138 00:04:58,919 --> 00:05:03,780 find your everyday life like shapes 139 00:05:00,419 --> 00:05:06,419 lines points you can abstract over this 140 00:05:03,780 --> 00:05:07,620 representation using grads where the 141 00:05:06,419 --> 00:05:10,560 shape of things no longer really 142 00:05:07,620 --> 00:05:12,479 mattered only the connectivity of the 143 00:05:10,560 --> 00:05:14,880 objects you care about matter 144 00:05:12,479 --> 00:05:18,240 2D data also include things like images 145 00:05:14,880 --> 00:05:21,120 and raster where things are recorded in 146 00:05:18,240 --> 00:05:23,759 pixels so for each pixel it represents 147 00:05:21,120 --> 00:05:25,860 some actual real-life geographic area 148 00:05:23,759 --> 00:05:27,780 and for that area you measure some 149 00:05:25,860 --> 00:05:29,759 measurements maybe the amount of 150 00:05:27,780 --> 00:05:31,080 rainfall the amount of solar potential 151 00:05:29,759 --> 00:05:33,419 and so on 152 00:05:31,080 --> 00:05:35,520 so this is probably the simplest Vector 153 00:05:33,419 --> 00:05:38,340 data that you can think of it's just a 154 00:05:35,520 --> 00:05:40,080 triangle and a swear but you may be able 155 00:05:38,340 --> 00:05:42,660 to do more interesting to them when you 156 00:05:40,080 --> 00:05:44,580 start moving them around rotating them 157 00:05:42,660 --> 00:05:46,020 finding the intersection between them 158 00:05:44,580 --> 00:05:48,300 which is highlighted in purple here 159 00:05:46,020 --> 00:05:50,820 you're going to merge them you'll use 160 00:05:48,300 --> 00:05:53,220 one shape to match the other shapes 161 00:05:50,820 --> 00:05:54,720 solve these are common operations that 162 00:05:53,220 --> 00:05:56,759 you may think of when we're dealing with 163 00:05:54,720 --> 00:05:58,740 Vector data 164 00:05:56,759 --> 00:06:00,960 and you can think of this representation 165 00:05:58,740 --> 00:06:02,820 as not unique to geospatial data alone 166 00:06:00,960 --> 00:06:05,160 because they are shapes they are 167 00:06:02,820 --> 00:06:06,840 practically everywhere they reduce a lot 168 00:06:05,160 --> 00:06:09,300 in computer Graphics as well so there 169 00:06:06,840 --> 00:06:10,860 are a lot of literature and algorithms 170 00:06:09,300 --> 00:06:12,539 that have been developed over the years 171 00:06:10,860 --> 00:06:14,340 that we can leverage file geospatial 172 00:06:12,539 --> 00:06:17,400 data analysis 173 00:06:14,340 --> 00:06:18,780 speaking of tools and algorithms at my 174 00:06:17,400 --> 00:06:22,319 team we use 175 00:06:18,780 --> 00:06:24,600 um very common Suite tools to extract 176 00:06:22,319 --> 00:06:27,360 insights from your spatial data the 177 00:06:24,600 --> 00:06:29,819 favorites are qgis jio pandas Chef Lee 178 00:06:27,360 --> 00:06:32,699 and network X which we will go over 179 00:06:29,819 --> 00:06:34,860 shortly we also use raster IO and 180 00:06:32,699 --> 00:06:36,960 lastify which are libraries for raster 181 00:06:34,860 --> 00:06:38,460 and lighter data but we will not talk 182 00:06:36,960 --> 00:06:40,080 about in this presentation because we 183 00:06:38,460 --> 00:06:42,780 don't have enough time 184 00:06:40,080 --> 00:06:45,360 so the first stop is qgis 185 00:06:42,780 --> 00:06:47,039 this where when we receive data from a 186 00:06:45,360 --> 00:06:49,139 client or we download data from the 187 00:06:47,039 --> 00:06:50,819 internet we will usually just plug them 188 00:06:49,139 --> 00:06:52,800 into this tool and it will plot 189 00:06:50,819 --> 00:06:55,199 everything onto the map and then we can 190 00:06:52,800 --> 00:06:57,060 scroll around just learn Google Maps we 191 00:06:55,199 --> 00:06:58,020 can also click on the actual shapes 192 00:06:57,060 --> 00:06:59,880 itself 193 00:06:58,020 --> 00:07:02,759 to fight attributes associated with that 194 00:06:59,880 --> 00:07:05,340 [ __ ] so here we are having a lot of 195 00:07:02,759 --> 00:07:07,080 polygons and orange shapes that are the 196 00:07:05,340 --> 00:07:09,720 footprints of houses and we also have 197 00:07:07,080 --> 00:07:10,680 dots that represent 198 00:07:09,720 --> 00:07:12,600 um 199 00:07:10,680 --> 00:07:15,419 points of interest 200 00:07:12,600 --> 00:07:17,100 and qgis also have a lot of tools 201 00:07:15,419 --> 00:07:19,319 associated with it it's a full power 202 00:07:17,100 --> 00:07:21,000 Analytics tool so it can also do all 203 00:07:19,319 --> 00:07:23,400 that geometric operations that I talk 204 00:07:21,000 --> 00:07:26,400 about it can merge things find 205 00:07:23,400 --> 00:07:29,220 geometries of Interest as an open source 206 00:07:26,400 --> 00:07:31,319 tool qgis also has a lot of third-party 207 00:07:29,220 --> 00:07:33,060 plugins that you can use to improve your 208 00:07:31,319 --> 00:07:34,560 analytics workflow and you can actually 209 00:07:33,060 --> 00:07:36,300 write Python scripts that can run in 210 00:07:34,560 --> 00:07:38,639 qgis as well 211 00:07:36,300 --> 00:07:40,139 if we have determined that qgis is not 212 00:07:38,639 --> 00:07:41,880 sufficient for our work and we really 213 00:07:40,139 --> 00:07:43,680 want to dig deep into things and start 214 00:07:41,880 --> 00:07:46,080 automating things with code the second 215 00:07:43,680 --> 00:07:47,580 stop is usually geopanders it's kind of 216 00:07:46,080 --> 00:07:50,099 like pandas which is a library for 217 00:07:47,580 --> 00:07:52,560 manipulating tabular data except we have 218 00:07:50,099 --> 00:07:55,800 an additional geometry column 219 00:07:52,560 --> 00:07:58,620 and it can read and work with the most 220 00:07:55,800 --> 00:08:01,560 common geospatial data formats like SHP 221 00:07:58,620 --> 00:08:03,900 LG or Json or GDP so in this code 222 00:08:01,560 --> 00:08:07,199 snippet on the right I'm importing your 223 00:08:03,900 --> 00:08:09,300 pandas and using the read5 API to read 224 00:08:07,199 --> 00:08:12,300 all the geometries for suburbs in South 225 00:08:09,300 --> 00:08:16,259 Australia and this read file API is very 226 00:08:12,300 --> 00:08:18,000 similar to the pandas read CSV API and 227 00:08:16,259 --> 00:08:19,919 you can pass arguments to it to specify 228 00:08:18,000 --> 00:08:22,560 which format you want to read it 229 00:08:19,919 --> 00:08:24,300 and the next line of code to CIS is 230 00:08:22,560 --> 00:08:27,539 where the geometry column really shines 231 00:08:24,300 --> 00:08:29,220 here I'm using the 2cis API to perform 232 00:08:27,539 --> 00:08:31,319 an operation that transforms one 233 00:08:29,220 --> 00:08:32,700 coordinate system to another so when you 234 00:08:31,319 --> 00:08:34,380 think of coordinate system you probably 235 00:08:32,700 --> 00:08:36,599 think of like latitude and longitude 236 00:08:34,380 --> 00:08:38,580 that can locate anywhere on the globe 237 00:08:36,599 --> 00:08:40,380 but it's not the only coordinate system 238 00:08:38,580 --> 00:08:42,360 and in fact it's not even useful for 239 00:08:40,380 --> 00:08:44,940 some use cases where we want to measure 240 00:08:42,360 --> 00:08:46,920 precise distances in meters you can't 241 00:08:44,940 --> 00:08:48,959 really have a distance in degrees it 242 00:08:46,920 --> 00:08:51,060 doesn't really make a lot of sense so 243 00:08:48,959 --> 00:08:53,940 what we do is we'll convert that corner 244 00:08:51,060 --> 00:08:56,640 system into one that is locally precise 245 00:08:53,940 --> 00:08:58,440 for South Australia where we can measure 246 00:08:56,640 --> 00:09:00,360 distances in meters and so that's what 247 00:08:58,440 --> 00:09:03,660 we're doing with this API call 248 00:09:00,360 --> 00:09:06,720 and then we can perform common pandas 249 00:09:03,660 --> 00:09:09,660 operations so things that can work on 250 00:09:06,720 --> 00:09:11,519 tables look similar here here I'm just 251 00:09:09,660 --> 00:09:13,800 indexing some columns and showing it in 252 00:09:11,519 --> 00:09:15,959 my tributor notebook 253 00:09:13,800 --> 00:09:17,820 if we want to go a bit further and 254 00:09:15,959 --> 00:09:20,580 directly manipulate the geometry itself 255 00:09:17,820 --> 00:09:23,220 we will turn to Shipley Shipley actually 256 00:09:20,580 --> 00:09:25,920 underlies of the geopanders geometries 257 00:09:23,220 --> 00:09:28,980 objects so when I print the typed of the 258 00:09:25,920 --> 00:09:30,600 first geometry in the geometry column it 259 00:09:28,980 --> 00:09:32,700 actually comes out as shapley the 260 00:09:30,600 --> 00:09:35,399 geometry the polygon 261 00:09:32,700 --> 00:09:37,080 and I can also use a lot of the common 262 00:09:35,399 --> 00:09:38,820 geometry operations that Chevrolet 263 00:09:37,080 --> 00:09:41,640 provides and here I'm trying to use the 264 00:09:38,820 --> 00:09:43,440 unary union operation and what this does 265 00:09:41,640 --> 00:09:44,459 is it just merges all the geometries 266 00:09:43,440 --> 00:09:46,560 together 267 00:09:44,459 --> 00:09:48,779 and because submerging the geometries of 268 00:09:46,560 --> 00:09:50,580 suburbs in South Australia I'd expect to 269 00:09:48,779 --> 00:09:54,060 see what the state of South Australia to 270 00:09:50,580 --> 00:09:56,640 emerge and it kind of does except 271 00:09:54,060 --> 00:09:58,440 there's this big hole to the left which 272 00:09:56,640 --> 00:10:00,000 I don't really know why maybe there's 273 00:09:58,440 --> 00:10:03,720 just no one living there 274 00:10:00,000 --> 00:10:05,339 but um as an old disciplines of data 275 00:10:03,720 --> 00:10:07,580 science data quality is probably one of 276 00:10:05,339 --> 00:10:10,320 the biggest problem that we have and 277 00:10:07,580 --> 00:10:12,660 geospatial data is now stranger to that 278 00:10:10,320 --> 00:10:15,720 problem as well 279 00:10:12,660 --> 00:10:17,279 to really show how powerful these 280 00:10:15,720 --> 00:10:20,339 libraries are though we need to really 281 00:10:17,279 --> 00:10:21,720 dig into the use cases and the projects 282 00:10:20,339 --> 00:10:23,160 that we work with our clients to bring 283 00:10:21,720 --> 00:10:24,779 value to them 284 00:10:23,160 --> 00:10:26,339 so the first project I would like to 285 00:10:24,779 --> 00:10:27,600 share with you is search pipe 286 00:10:26,339 --> 00:10:30,660 maintenance 287 00:10:27,600 --> 00:10:32,640 so the problem that we're trying to 288 00:10:30,660 --> 00:10:34,860 solve for our clients here which is a 289 00:10:32,640 --> 00:10:36,899 water utility company in New South Wales 290 00:10:34,860 --> 00:10:38,820 is to optimize the sewage means 291 00:10:36,899 --> 00:10:40,620 maintenance program to achieve the best 292 00:10:38,820 --> 00:10:43,200 performance for their budget 293 00:10:40,620 --> 00:10:47,220 so the idea is sewage pipe 294 00:10:43,200 --> 00:10:48,839 sdh they get more prone to breakage or 295 00:10:47,220 --> 00:10:50,640 leakages and it releases a lot of 296 00:10:48,839 --> 00:10:53,279 harmful things environment and also 297 00:10:50,640 --> 00:10:55,019 cause a ton to fix up so ideally we 298 00:10:53,279 --> 00:10:57,839 would like to replace or maintain the 299 00:10:55,019 --> 00:10:59,160 pipes before it gets too old 300 00:10:57,839 --> 00:11:01,620 um obviously you cannot really replace 301 00:10:59,160 --> 00:11:04,019 everything everywhere all at once so you 302 00:11:01,620 --> 00:11:06,839 have to make decisions on which pipe to 303 00:11:04,019 --> 00:11:09,540 replace now within your budget 304 00:11:06,839 --> 00:11:12,000 so for the scope of this work we want to 305 00:11:09,540 --> 00:11:14,399 automate this automation optimization 306 00:11:12,000 --> 00:11:16,560 program and we also want to add some new 307 00:11:14,399 --> 00:11:19,200 additional features we want to tell the 308 00:11:16,560 --> 00:11:21,360 client how accessible a pipe is 309 00:11:19,200 --> 00:11:24,000 to maintenance equipment and this kind 310 00:11:21,360 --> 00:11:27,120 of breaks down into two sub-measures one 311 00:11:24,000 --> 00:11:29,519 is how much of a pipe is built over Say 312 00:11:27,120 --> 00:11:31,620 by property so you can see the pipe 313 00:11:29,519 --> 00:11:33,480 highlighted in red is kind of built over 314 00:11:31,620 --> 00:11:35,760 in certain places 315 00:11:33,480 --> 00:11:38,040 the second dimension is how accessible a 316 00:11:35,760 --> 00:11:40,500 pipe is from the road behind like all 317 00:11:38,040 --> 00:11:42,300 these properties so let's say I want to 318 00:11:40,500 --> 00:11:44,760 access the red pipe 319 00:11:42,300 --> 00:11:47,279 from the road then I may need to go over 320 00:11:44,760 --> 00:11:49,200 like the building in Gray and the 321 00:11:47,279 --> 00:11:50,700 building May does not have a large 322 00:11:49,200 --> 00:11:52,680 enough right way for my equipment to 323 00:11:50,700 --> 00:11:55,680 drive through 324 00:11:52,680 --> 00:11:59,100 so for this problem we have data that 325 00:11:55,680 --> 00:12:01,440 are the pipes and lines we also have the 326 00:11:59,100 --> 00:12:03,720 outlines of all the buildings and these 327 00:12:01,440 --> 00:12:05,760 orange polygons 328 00:12:03,720 --> 00:12:07,980 and these are actually generated from 329 00:12:05,760 --> 00:12:09,300 satellite images using the Deep learning 330 00:12:07,980 --> 00:12:10,800 model so that's an interesting 331 00:12:09,300 --> 00:12:13,740 application there 332 00:12:10,800 --> 00:12:16,200 and finally we have some outlines of the 333 00:12:13,740 --> 00:12:18,060 lots for each property and this data is 334 00:12:16,200 --> 00:12:20,220 publicly available from the new software 335 00:12:18,060 --> 00:12:22,560 government 336 00:12:20,220 --> 00:12:24,540 so to stop the first measure which is 337 00:12:22,560 --> 00:12:27,899 how much of a pipe is spills over the 338 00:12:24,540 --> 00:12:31,260 algorithm is relatively simple 339 00:12:27,899 --> 00:12:32,700 um intuitively you can think of well if 340 00:12:31,260 --> 00:12:34,920 I want to find how much of a pipe is 341 00:12:32,700 --> 00:12:36,600 built over I just find the exact section 342 00:12:34,920 --> 00:12:37,680 of pipe that is underneath any other 343 00:12:36,600 --> 00:12:40,079 geometry 344 00:12:37,680 --> 00:12:40,920 and this is a geometry intersection 345 00:12:40,079 --> 00:12:43,200 problem 346 00:12:40,920 --> 00:12:46,200 and then Shipley it can do that with a 347 00:12:43,200 --> 00:12:49,079 single API call the intersection API 348 00:12:46,200 --> 00:12:50,940 so energy geometry have an intersection 349 00:12:49,079 --> 00:12:53,100 method that can be called on knowledge 350 00:12:50,940 --> 00:12:55,260 geometry and then Shipley will handle 351 00:12:53,100 --> 00:12:56,820 the specific algorithms to intersect a 352 00:12:55,260 --> 00:12:58,019 line with the line or a live with the 353 00:12:56,820 --> 00:12:58,980 polygon 354 00:12:58,019 --> 00:13:00,839 Etc 355 00:12:58,980 --> 00:13:03,180 and it's all well and good 356 00:13:00,839 --> 00:13:04,579 however the problem of determining how 357 00:13:03,180 --> 00:13:07,019 much of the pipe is acceptable 358 00:13:04,579 --> 00:13:09,300 accessible for an equipment behind 359 00:13:07,019 --> 00:13:12,300 properties a bit harder there's no 360 00:13:09,300 --> 00:13:14,700 really single unique operation that you 361 00:13:12,300 --> 00:13:17,639 can just do and get the answer 362 00:13:14,700 --> 00:13:19,980 so our first naive approach is to try to 363 00:13:17,639 --> 00:13:22,800 measure the gap between the building and 364 00:13:19,980 --> 00:13:24,480 the large fence line so the idea is well 365 00:13:22,800 --> 00:13:26,760 if the pipe is in front of the property 366 00:13:24,480 --> 00:13:29,160 then surely is accessible so that's the 367 00:13:26,760 --> 00:13:32,940 trivial case if it's behind the property 368 00:13:29,160 --> 00:13:34,980 then the issue is is there like a big 369 00:13:32,940 --> 00:13:37,620 enough gap between the building and a 370 00:13:34,980 --> 00:13:39,600 lot of fence line for the equipment to 371 00:13:37,620 --> 00:13:41,459 drive straight in like a driveway that's 372 00:13:39,600 --> 00:13:43,800 not blocked by your garage 373 00:13:41,459 --> 00:13:45,600 so visually we're trying to measure the 374 00:13:43,800 --> 00:13:47,339 distance of that red line between the 375 00:13:45,600 --> 00:13:50,579 building and the lot 376 00:13:47,339 --> 00:13:52,500 and in shortly distance is very easily 377 00:13:50,579 --> 00:13:54,060 performed consuming you are in the 378 00:13:52,500 --> 00:13:56,160 projected Corner system that I mentioned 379 00:13:54,060 --> 00:13:58,680 earlier you can use a distant API 380 00:13:56,160 --> 00:14:00,839 between any two geometries again and 381 00:13:58,680 --> 00:14:02,220 Shipley will handle all the specifics to 382 00:14:00,839 --> 00:14:06,000 measure the shortest distance between 383 00:14:02,220 --> 00:14:07,980 any two point on those two geometries 384 00:14:06,000 --> 00:14:09,959 this approach however kind of just 385 00:14:07,980 --> 00:14:12,959 breaks down when you add more complexity 386 00:14:09,959 --> 00:14:14,700 to the problem so here I'm trying to add 387 00:14:12,959 --> 00:14:18,899 some extra like 388 00:14:14,700 --> 00:14:20,940 sheds and garage and maybe a poles so 389 00:14:18,899 --> 00:14:23,760 there's extra polygons now in the lot 390 00:14:20,940 --> 00:14:24,899 and so this minimum distance between the 391 00:14:23,760 --> 00:14:27,779 building and a lot of fence light 392 00:14:24,899 --> 00:14:29,399 doesn't really tell the whole story 393 00:14:27,779 --> 00:14:31,380 because maybe there's sufficient 394 00:14:29,399 --> 00:14:34,200 distance but a shared filter data lines 395 00:14:31,380 --> 00:14:36,240 blocking as you can see or maybe 396 00:14:34,200 --> 00:14:37,920 so now you have to include the distance 397 00:14:36,240 --> 00:14:39,779 between the building and the shed as 398 00:14:37,920 --> 00:14:41,339 well but if that distance is too small 399 00:14:39,779 --> 00:14:42,720 maybe you also need to consider the 400 00:14:41,339 --> 00:14:44,100 first distance that we measured because 401 00:14:42,720 --> 00:14:45,060 there may still be a path straight 402 00:14:44,100 --> 00:14:47,279 through 403 00:14:45,060 --> 00:14:48,779 so I guess you can come up with some 404 00:14:47,279 --> 00:14:51,180 algorithms to iterate over all these 405 00:14:48,779 --> 00:14:53,639 combinations of distances but it's not 406 00:14:51,180 --> 00:14:55,139 very naive algorithm anymore and it's 407 00:14:53,639 --> 00:14:57,660 starting to be very complex and error 408 00:14:55,139 --> 00:14:59,839 prone fortunately there is a better 409 00:14:57,660 --> 00:15:02,779 approach which takes which uses 410 00:14:59,839 --> 00:15:06,000 buffering and buffering is just 411 00:15:02,779 --> 00:15:07,560 expanding or shrinking a geometry by a 412 00:15:06,000 --> 00:15:10,079 certain set distance 413 00:15:07,560 --> 00:15:12,560 so in this illustration we are buffering 414 00:15:10,079 --> 00:15:15,839 expanding the original building 415 00:15:12,560 --> 00:15:18,600 geometries by one meter so you can see 416 00:15:15,839 --> 00:15:20,940 the original geometries in Orange with 417 00:15:18,600 --> 00:15:23,760 the X cross on it 418 00:15:20,940 --> 00:15:25,620 and a buffered outlines in a more 419 00:15:23,760 --> 00:15:28,019 transparent orange color 420 00:15:25,620 --> 00:15:30,420 I'm also shrinking the original large 421 00:15:28,019 --> 00:15:32,519 geometry by one meter and the remaining 422 00:15:30,420 --> 00:15:34,680 space is color in green 423 00:15:32,519 --> 00:15:37,079 so the inside of this approach is that 424 00:15:34,680 --> 00:15:39,060 if we make some simplifying assumption 425 00:15:37,079 --> 00:15:41,339 on the shape of the equipment 426 00:15:39,060 --> 00:15:44,160 that we care about and we're just gonna 427 00:15:41,339 --> 00:15:46,680 see there is a sphere of some radius and 428 00:15:44,160 --> 00:15:48,000 we can also tweak this radius to 429 00:15:46,680 --> 00:15:49,019 actually account for the actual shape of 430 00:15:48,000 --> 00:15:51,600 the equipment 431 00:15:49,019 --> 00:15:53,399 than the center of the sphere must lie 432 00:15:51,600 --> 00:15:55,699 in the green area and you can kind of 433 00:15:53,399 --> 00:15:58,699 see this for yourself if you try to like 434 00:15:55,699 --> 00:16:00,720 imagine placing the center of the sphere 435 00:15:58,699 --> 00:16:02,940 anywhere that is not in the green area 436 00:16:00,720 --> 00:16:05,040 then when you draw the sphere it must 437 00:16:02,940 --> 00:16:07,320 intersect the large fence line on the 438 00:16:05,040 --> 00:16:09,180 original building geometry somewhere 439 00:16:07,320 --> 00:16:11,220 because the radius would exceed the 440 00:16:09,180 --> 00:16:14,760 buffer distance 441 00:16:11,220 --> 00:16:17,760 so if we now 442 00:16:14,760 --> 00:16:19,860 check if the green area contains both 443 00:16:17,760 --> 00:16:22,079 the pipe and access to the road then 444 00:16:19,860 --> 00:16:23,760 surely some equipment that is a sphere 445 00:16:22,079 --> 00:16:26,220 would be able to go from the road to the 446 00:16:23,760 --> 00:16:29,160 pipe we cannot do this in shiftly using 447 00:16:26,220 --> 00:16:31,079 the difference and buffer API which does 448 00:16:29,160 --> 00:16:33,480 exactly what they say it does 449 00:16:31,079 --> 00:16:35,519 we compute the difference between the 450 00:16:33,480 --> 00:16:38,220 negatively buffer a lot so that's the 451 00:16:35,519 --> 00:16:40,019 strength lot and the positively buffer 452 00:16:38,220 --> 00:16:41,279 buildings envelope to give the green 453 00:16:40,019 --> 00:16:43,680 area 454 00:16:41,279 --> 00:16:45,480 we then iterate through each polygons 455 00:16:43,680 --> 00:16:47,399 within that green area just in case that 456 00:16:45,480 --> 00:16:49,500 the buildings went above for drops the 457 00:16:47,399 --> 00:16:51,480 area in half and for each of that 458 00:16:49,500 --> 00:16:54,540 polygon you can check if it intersects 459 00:16:51,480 --> 00:16:57,060 the road and the pipe geometries if it 460 00:16:54,540 --> 00:16:59,519 does then the pipe is accessible 461 00:16:57,060 --> 00:17:01,560 the outcomes of all of this algorithm is 462 00:16:59,519 --> 00:17:03,480 some condensed summarization 463 00:17:01,560 --> 00:17:05,040 measurements for H Pi let's say how much 464 00:17:03,480 --> 00:17:06,660 of it's built over and how much of it is 465 00:17:05,040 --> 00:17:08,220 accessible 466 00:17:06,660 --> 00:17:10,319 and all of these measurements can then 467 00:17:08,220 --> 00:17:12,660 be fed to Downstream applications for 468 00:17:10,319 --> 00:17:15,600 example like in the dashboard or in some 469 00:17:12,660 --> 00:17:17,459 reporting program so that the client can 470 00:17:15,600 --> 00:17:19,439 make decisions or it can even be fed 471 00:17:17,459 --> 00:17:20,699 into another machine learning program to 472 00:17:19,439 --> 00:17:22,500 actually perform that optimization 473 00:17:20,699 --> 00:17:24,720 program in the first place 474 00:17:22,500 --> 00:17:26,939 sometimes though we don't really want to 475 00:17:24,720 --> 00:17:29,580 have this summarization measurements 476 00:17:26,939 --> 00:17:32,040 from a geospatial data we instead want 477 00:17:29,580 --> 00:17:34,200 some geospatial data as outputs from our 478 00:17:32,040 --> 00:17:35,820 initial geospatial data because we may 479 00:17:34,200 --> 00:17:36,900 want to for example visualize them on 480 00:17:35,820 --> 00:17:38,460 the map 481 00:17:36,900 --> 00:17:41,520 and the second project I want to share 482 00:17:38,460 --> 00:17:43,440 with you is one where this is the case 483 00:17:41,520 --> 00:17:44,340 in this project heavy vehicle load 484 00:17:43,440 --> 00:17:47,340 access 485 00:17:44,340 --> 00:17:47,340 sorry 486 00:17:48,600 --> 00:17:53,100 the the problem is we want to evaluate 487 00:17:51,240 --> 00:17:56,220 Road networks in New South Wales against 488 00:17:53,100 --> 00:17:58,200 electric trucks Dimensions so the idea 489 00:17:56,220 --> 00:18:00,240 is the state of New South Wales wants to 490 00:17:58,200 --> 00:18:02,400 adopt electric trucks but these are 491 00:18:00,240 --> 00:18:04,200 really large compared to original trucks 492 00:18:02,400 --> 00:18:06,000 they are larger and longer to account 493 00:18:04,200 --> 00:18:08,820 for their batteries and you have ever 494 00:18:06,000 --> 00:18:10,080 driven behind trucks you know that you 495 00:18:08,820 --> 00:18:12,900 don't want to be 496 00:18:10,080 --> 00:18:15,059 in size turning curve because as it 497 00:18:12,900 --> 00:18:17,520 turns it create this really large web 498 00:18:15,059 --> 00:18:19,799 path which is this total area that it 499 00:18:17,520 --> 00:18:21,059 Trace out as it turns and you don't want 500 00:18:19,799 --> 00:18:22,500 to be inside it because then it's going 501 00:18:21,059 --> 00:18:24,179 to hit you and that's not going to be a 502 00:18:22,500 --> 00:18:25,919 fun experience 503 00:18:24,179 --> 00:18:27,240 so the client wants to understand if the 504 00:18:25,919 --> 00:18:29,820 road networks in New South Wales is 505 00:18:27,240 --> 00:18:32,460 sufficient for all these new vehicles 506 00:18:29,820 --> 00:18:34,559 so for that purpose we 507 00:18:32,460 --> 00:18:39,200 yep sorry 508 00:18:34,559 --> 00:18:39,200 I think my laptop is not liking me today 509 00:18:39,960 --> 00:18:42,500 yep 510 00:18:48,780 --> 00:18:54,120 yep so we have developed a simple 511 00:18:51,720 --> 00:18:56,760 algorithm that can simulate the swept 512 00:18:54,120 --> 00:18:58,260 path given the path that the Mover of 513 00:18:56,760 --> 00:19:00,539 the vehicle takes 514 00:18:58,260 --> 00:19:02,640 but this is actually the start of a new 515 00:19:00,539 --> 00:19:03,960 problem because this simulation 516 00:19:02,640 --> 00:19:06,059 algorithm 517 00:19:03,960 --> 00:19:08,220 needs the path that the Mover takes but 518 00:19:06,059 --> 00:19:10,740 the data that we were given actually do 519 00:19:08,220 --> 00:19:12,419 not have any information on what actual 520 00:19:10,740 --> 00:19:15,240 pass it can take to turn 521 00:19:12,419 --> 00:19:17,880 so the information we're given is a 522 00:19:15,240 --> 00:19:19,200 single line for each Road geometry that 523 00:19:17,880 --> 00:19:21,660 starts and end where the road starts and 524 00:19:19,200 --> 00:19:24,600 end so in this intersection we have many 525 00:19:21,660 --> 00:19:28,740 separate roads but we have no 526 00:19:24,600 --> 00:19:31,940 information that tells the algorithm 527 00:19:28,740 --> 00:19:31,940 that it can turn 528 00:19:42,179 --> 00:19:47,520 that you can take Row one two and three 529 00:19:44,760 --> 00:19:49,260 to turn right from the top so none of 530 00:19:47,520 --> 00:19:51,780 that information is available to the 531 00:19:49,260 --> 00:19:54,299 algorithm itself all it says is this 532 00:19:51,780 --> 00:19:56,220 separate collections of Roads 533 00:19:54,299 --> 00:19:58,020 so we need a way to given this 534 00:19:56,220 --> 00:20:00,480 collection of Roads it numerates the 535 00:19:58,020 --> 00:20:03,660 possible hats that the truck can take 536 00:20:00,480 --> 00:20:06,360 and then output some actual Road 537 00:20:03,660 --> 00:20:09,840 geometry that the truck can the 538 00:20:06,360 --> 00:20:15,020 simulation algorithm that can then take 539 00:20:09,840 --> 00:20:15,020 yeah sorry for that all right 540 00:20:20,039 --> 00:20:24,419 um so the way that we solved It Is by 541 00:20:23,400 --> 00:20:27,660 using 542 00:20:24,419 --> 00:20:29,640 uh Network X Library which constructs 543 00:20:27,660 --> 00:20:31,980 graph 544 00:20:29,640 --> 00:20:33,419 okay so let's see if I can get this to 545 00:20:31,980 --> 00:20:35,280 work now 546 00:20:33,419 --> 00:20:36,900 so what we did is we take each 547 00:20:35,280 --> 00:20:39,679 individual Road 548 00:20:36,900 --> 00:20:39,679 and turn it 549 00:20:41,880 --> 00:20:48,120 just go to the last yeah and don't click 550 00:20:44,760 --> 00:20:50,940 we take each individual Road and compute 551 00:20:48,120 --> 00:20:52,200 and treat it as an abstract Edge so the 552 00:20:50,940 --> 00:20:54,780 start and end of the road will be the 553 00:20:52,200 --> 00:20:56,160 starting n vertex in the graph and we 554 00:20:54,780 --> 00:20:59,340 ignore all the actual coordinates that 555 00:20:56,160 --> 00:21:01,679 link those two vertex together 556 00:20:59,340 --> 00:21:03,900 then if we Define some start and 557 00:21:01,679 --> 00:21:06,539 entrance vertices to the graph to the 558 00:21:03,900 --> 00:21:08,340 intersection so maybe you can use if 559 00:21:06,539 --> 00:21:10,919 there's only one Edge to the node then 560 00:21:08,340 --> 00:21:12,780 that is an entrance on exit vertex then 561 00:21:10,919 --> 00:21:14,340 we can use a graph algorithm to kind of 562 00:21:12,780 --> 00:21:16,500 find the shortest path between those two 563 00:21:14,340 --> 00:21:19,080 vertex and that will give us a list of 564 00:21:16,500 --> 00:21:21,360 edges in this graph 565 00:21:19,080 --> 00:21:23,340 and then we can extract the coordinates 566 00:21:21,360 --> 00:21:25,260 of the road that is associated with the 567 00:21:23,340 --> 00:21:27,600 edge and then concatenate them and that 568 00:21:25,260 --> 00:21:28,740 will form the final Road the truck can 569 00:21:27,600 --> 00:21:31,280 take 570 00:21:28,740 --> 00:21:31,280 so 571 00:21:33,659 --> 00:21:36,140 yep 572 00:21:38,700 --> 00:21:43,200 when in networks X this is done using 573 00:21:41,159 --> 00:21:44,940 the code in the bottom right so we first 574 00:21:43,200 --> 00:21:46,919 initialize this graph component this 575 00:21:44,940 --> 00:21:48,480 graph class then we iterate through each 576 00:21:46,919 --> 00:21:50,940 geometries 577 00:21:48,480 --> 00:21:52,860 and then we can index into the core in 578 00:21:50,940 --> 00:21:54,840 this array and find the first and final 579 00:21:52,860 --> 00:21:57,659 coordinate entry let us start an N 580 00:21:54,840 --> 00:22:01,380 vertex Network X actually allows you to 581 00:21:57,659 --> 00:22:03,780 input any hashable immutable python data 582 00:22:01,380 --> 00:22:06,120 structure to be the vertex so you can 583 00:22:03,780 --> 00:22:07,559 just insert the coordinates pair right 584 00:22:06,120 --> 00:22:09,299 into it 585 00:22:07,559 --> 00:22:11,760 and we can also attach the coordinates 586 00:22:09,299 --> 00:22:13,200 of the original live stream geometry as 587 00:22:11,760 --> 00:22:15,539 well and just make it easy makes it 588 00:22:13,200 --> 00:22:17,760 easier to just extract the coordinates 589 00:22:15,539 --> 00:22:21,120 and then concatenate all the 590 00:22:17,760 --> 00:22:23,039 the the paths together 591 00:22:21,120 --> 00:22:25,380 we can then smooth out this path using 592 00:22:23,039 --> 00:22:28,380 some additional numpy geometries numpy 593 00:22:25,380 --> 00:22:31,740 vectors and the math to create this red 594 00:22:28,380 --> 00:22:33,299 geometry that informs the simulation 595 00:22:31,740 --> 00:22:36,120 algorithm that the truck can now take 596 00:22:33,299 --> 00:22:38,100 this path to turn right and we can do 597 00:22:36,120 --> 00:22:40,500 the same for every other pairs or 598 00:22:38,100 --> 00:22:43,080 starting inverses vertex 599 00:22:40,500 --> 00:22:45,539 and that helps create all this paths 600 00:22:43,080 --> 00:22:48,120 that navigate this intersection 601 00:22:45,539 --> 00:22:51,000 now if I click next it will jumps all 602 00:22:48,120 --> 00:22:53,760 the way again so 603 00:22:51,000 --> 00:22:56,400 I will just present without any visual 604 00:22:53,760 --> 00:22:58,320 aid before passing it onto Christine so 605 00:22:56,400 --> 00:23:01,380 the output of this algorithm would be 606 00:22:58,320 --> 00:23:04,080 this swap pad geometry that we have 607 00:23:01,380 --> 00:23:06,539 simulated for the truck and then we can 608 00:23:04,080 --> 00:23:07,919 kind of look into the widths along the 609 00:23:06,539 --> 00:23:09,659 square path 610 00:23:07,919 --> 00:23:12,480 and then we look at the width of the 611 00:23:09,659 --> 00:23:14,700 road and then we say oh at this location 612 00:23:12,480 --> 00:23:16,860 the width of this web hat is too large 613 00:23:14,700 --> 00:23:18,780 and so there will be a problem because 614 00:23:16,860 --> 00:23:20,460 this drug will now go into the opposite 615 00:23:18,780 --> 00:23:21,659 lane and cause problems for other 616 00:23:20,460 --> 00:23:23,340 vehicles 617 00:23:21,659 --> 00:23:27,120 oh it may intersect some roadside 618 00:23:23,340 --> 00:23:29,460 geometric assets like maybe fence or 619 00:23:27,120 --> 00:23:31,500 traffic lights I don't know 620 00:23:29,460 --> 00:23:33,900 the problem is we don't really have very 621 00:23:31,500 --> 00:23:36,539 good labels on how Wire Road is at any 622 00:23:33,900 --> 00:23:39,780 point it would be good if we have the 623 00:23:36,539 --> 00:23:42,900 actual geometries for the road like a 624 00:23:39,780 --> 00:23:45,240 polygon that covers the area of the road 625 00:23:42,900 --> 00:23:46,919 but that is actually quite hard to get 626 00:23:45,240 --> 00:23:49,559 so as in all other 627 00:23:46,919 --> 00:23:51,059 data science problems QT label is also a 628 00:23:49,559 --> 00:23:53,700 problem in our case where we don't have 629 00:23:51,059 --> 00:23:56,940 a truthy geometry labels that we can 630 00:23:53,700 --> 00:23:59,100 compare this webpad simulation with 631 00:23:56,940 --> 00:24:01,380 so with that said I will now pass it on 632 00:23:59,100 --> 00:24:03,780 to Christine who will take it over from 633 00:24:01,380 --> 00:24:06,140 here so yeah and it's to move to the 634 00:24:03,780 --> 00:24:06,140 next slide 635 00:24:07,440 --> 00:24:13,200 yes so with this last use case 636 00:24:10,799 --> 00:24:14,940 um we are pushing the libraries and 637 00:24:13,200 --> 00:24:16,140 tools that long was talking about a 638 00:24:14,940 --> 00:24:17,760 little bit to the site they're not the 639 00:24:16,140 --> 00:24:19,740 center stage of it but they are still 640 00:24:17,760 --> 00:24:22,919 very important to produce inputs and 641 00:24:19,740 --> 00:24:25,620 outputs for our analysis our team 642 00:24:22,919 --> 00:24:29,220 frequently supports traffic modeling 643 00:24:25,620 --> 00:24:32,400 simulation tasks with data analytics and 644 00:24:29,220 --> 00:24:34,440 Automation and occasionally the tools 645 00:24:32,400 --> 00:24:37,440 and Frameworks that are in use by those 646 00:24:34,440 --> 00:24:39,600 teams are necessarily suitable for for 647 00:24:37,440 --> 00:24:41,039 the problem at hand and in this case one 648 00:24:39,600 --> 00:24:43,799 of the issues they encountered was 649 00:24:41,039 --> 00:24:46,559 modeling reversal of vehicles 650 00:24:43,799 --> 00:24:48,900 so what was the problem and we were 651 00:24:46,559 --> 00:24:51,120 asked to look at a Precinct in the 652 00:24:48,900 --> 00:24:53,039 mountainous region that gets frequented 653 00:24:51,120 --> 00:24:56,340 particularly on weekends by a lot of 654 00:24:53,039 --> 00:24:57,539 tourists and these Precinct is to access 655 00:24:56,340 --> 00:25:00,240 roads 656 00:24:57,539 --> 00:25:01,500 um that are very narrow and in a lot of 657 00:25:00,240 --> 00:25:03,179 places 658 00:25:01,500 --> 00:25:04,919 um they're just single files so you can 659 00:25:03,179 --> 00:25:06,840 imagine if you're driving down the road 660 00:25:04,919 --> 00:25:08,460 with a lot of traffic you're 661 00:25:06,840 --> 00:25:10,500 encountering other vehicles going the 662 00:25:08,460 --> 00:25:12,900 other direction quite frequently which 663 00:25:10,500 --> 00:25:15,539 causes you to either try to be brave and 664 00:25:12,900 --> 00:25:17,760 scramble past them or what is probably a 665 00:25:15,539 --> 00:25:19,260 little bit safer is to reverse and find 666 00:25:17,760 --> 00:25:21,480 a spot where you can safely let them 667 00:25:19,260 --> 00:25:23,700 pass 668 00:25:21,480 --> 00:25:25,679 The Operators of this road were 669 00:25:23,700 --> 00:25:27,480 particularly worried about like the 670 00:25:25,679 --> 00:25:29,100 latter people reversing up and down the 671 00:25:27,480 --> 00:25:30,779 road and creating too much traffic on it 672 00:25:29,100 --> 00:25:33,059 so they asked us to provide some 673 00:25:30,779 --> 00:25:35,220 foundations to intuitive traffic 674 00:25:33,059 --> 00:25:37,740 operations that they could use to 675 00:25:35,220 --> 00:25:39,480 alleviate that problem for example using 676 00:25:37,740 --> 00:25:41,220 shuttle buses to get the vast majority 677 00:25:39,480 --> 00:25:43,860 of people down the road with different 678 00:25:41,220 --> 00:25:45,779 schedules and different sizes or to 679 00:25:43,860 --> 00:25:48,500 implement boom gate Solutions on either 680 00:25:45,779 --> 00:25:48,500 end of the road 681 00:25:49,799 --> 00:25:55,020 um so how did we approach this problem 682 00:25:52,679 --> 00:25:56,220 we needed something that could iterate 683 00:25:55,020 --> 00:25:58,740 over a lot of different situations 684 00:25:56,220 --> 00:26:01,260 fairly quickly and 685 00:25:58,740 --> 00:26:03,960 um we decided to break the road up into 686 00:26:01,260 --> 00:26:05,940 cells and each of these cells would have 687 00:26:03,960 --> 00:26:08,580 properties such as whether they are 688 00:26:05,940 --> 00:26:10,440 single or double file and for example if 689 00:26:08,580 --> 00:26:12,260 the speed associated with traveling on 690 00:26:10,440 --> 00:26:15,500 this particular location 691 00:26:12,260 --> 00:26:18,059 and it will have a state which 692 00:26:15,500 --> 00:26:19,980 represents a vehicle being on the cell 693 00:26:18,059 --> 00:26:22,559 whether this vehicle is traveling east 694 00:26:19,980 --> 00:26:24,600 or west or whether it is currently 695 00:26:22,559 --> 00:26:27,419 reversing or in a holding State waiting 696 00:26:24,600 --> 00:26:29,159 for a vehicle to pass 697 00:26:27,419 --> 00:26:31,380 um for each time step in the simulation 698 00:26:29,159 --> 00:26:32,940 the cells would update their state in a 699 00:26:31,380 --> 00:26:34,799 random order based on the surrounding 700 00:26:32,940 --> 00:26:36,720 cells and we introduced some more 701 00:26:34,799 --> 00:26:39,500 variability for example the arrival 702 00:26:36,720 --> 00:26:41,700 times of cars at either end of that road 703 00:26:39,500 --> 00:26:43,320 based on distributions we got from 704 00:26:41,700 --> 00:26:46,279 traffic counts that were actually 705 00:26:43,320 --> 00:26:46,279 measured up front 706 00:26:46,380 --> 00:26:50,880 what allows us what this allows us to do 707 00:26:48,779 --> 00:26:54,120 is very quickly test different operating 708 00:26:50,880 --> 00:26:57,000 scenarios introduce the boom Gates and 709 00:26:54,120 --> 00:27:00,299 buses and other things and simulate 710 00:26:57,000 --> 00:27:03,440 thousands of typical days how traffic 711 00:27:00,299 --> 00:27:03,440 could evolve on this road 712 00:27:03,539 --> 00:27:08,039 how did we implement this at the core of 713 00:27:06,360 --> 00:27:10,140 this is a custom simulation that we 714 00:27:08,039 --> 00:27:12,000 wrote in Python and one of the key 715 00:27:10,140 --> 00:27:14,220 inputs to this is a shapefile that 716 00:27:12,000 --> 00:27:15,779 contains the geometry of the road and 717 00:27:14,220 --> 00:27:17,580 you can imagine that you can easily load 718 00:27:15,779 --> 00:27:19,559 it and then use shapely that Don was 719 00:27:17,580 --> 00:27:22,620 talking about before to break this road 720 00:27:19,559 --> 00:27:24,840 down into evenly sized segments 721 00:27:22,620 --> 00:27:27,120 in addition we put the vehicle counts in 722 00:27:24,840 --> 00:27:29,760 and the config file that then relates 723 00:27:27,120 --> 00:27:31,679 the change along that road and therefore 724 00:27:29,760 --> 00:27:33,659 it's relation to the cells with regards 725 00:27:31,679 --> 00:27:35,279 to what the speed limits are on or the 726 00:27:33,659 --> 00:27:36,779 average speeds on those sections of Road 727 00:27:35,279 --> 00:27:38,100 and which ones are the single file 728 00:27:36,779 --> 00:27:39,720 locations 729 00:27:38,100 --> 00:27:41,820 now once you've done that your 730 00:27:39,720 --> 00:27:45,059 simulation can essentially forget about 731 00:27:41,820 --> 00:27:46,919 the spatial context of this however it's 732 00:27:45,059 --> 00:27:48,900 really valuable to keep this information 733 00:27:46,919 --> 00:27:50,760 in the back of your head because what we 734 00:27:48,900 --> 00:27:53,340 did is we implemented the visualization 735 00:27:50,760 --> 00:27:55,200 in pi game for this which is basically 736 00:27:53,340 --> 00:27:57,480 translating the coordinates of the cells 737 00:27:55,200 --> 00:27:59,159 into screen coordinates and we could use 738 00:27:57,480 --> 00:28:01,140 this to take the client on our journey 739 00:27:59,159 --> 00:28:03,720 of developing developing the simulation 740 00:28:01,140 --> 00:28:05,700 and show them along the way our vehicles 741 00:28:03,720 --> 00:28:07,559 are actually moving along the Bendy 742 00:28:05,700 --> 00:28:09,480 Mountain Road 743 00:28:07,559 --> 00:28:11,039 then the simulation has a batch mode 744 00:28:09,480 --> 00:28:12,539 where you can run thousands of 745 00:28:11,039 --> 00:28:14,100 simulation and it's just chucking out 746 00:28:12,539 --> 00:28:15,240 files with the parameters that we are 747 00:28:14,100 --> 00:28:17,940 interested in 748 00:28:15,240 --> 00:28:19,799 so the last bit to this analysis this is 749 00:28:17,940 --> 00:28:22,020 missing is what do we do with our 750 00:28:19,799 --> 00:28:24,360 startup once simulation is produced it 751 00:28:22,020 --> 00:28:26,100 and in good old data science style we 752 00:28:24,360 --> 00:28:28,559 developed a python notebook that reads 753 00:28:26,100 --> 00:28:31,620 all of this in for single scenario and 754 00:28:28,559 --> 00:28:33,299 then creates aggregated outputs and 755 00:28:31,620 --> 00:28:34,440 again the spatial representation of 756 00:28:33,299 --> 00:28:36,419 these cells becomes really quite 757 00:28:34,440 --> 00:28:39,059 important because it allows us to 758 00:28:36,419 --> 00:28:41,279 produce a spatial representation of our 759 00:28:39,059 --> 00:28:44,640 results that then can be loaded for 760 00:28:41,279 --> 00:28:46,140 example in qgis and put on maps to show 761 00:28:44,640 --> 00:28:47,940 people the actual content of what's 762 00:28:46,140 --> 00:28:50,400 Happening 763 00:28:47,940 --> 00:28:52,740 and these are some example outputs I 764 00:28:50,400 --> 00:28:54,900 need to say that the y-axis went missing 765 00:28:52,740 --> 00:28:57,000 from these plots but it's a percentage 766 00:28:54,900 --> 00:28:59,220 of vehicles and it for example shows the 767 00:28:57,000 --> 00:29:01,260 travel time overall on the road the 768 00:28:59,220 --> 00:29:03,840 intergate travel time the number of 769 00:29:01,260 --> 00:29:06,480 interactions that vehicles have and on 770 00:29:03,840 --> 00:29:08,279 the left and the top right in particular 771 00:29:06,480 --> 00:29:10,860 on the left you can see the hot spots of 772 00:29:08,279 --> 00:29:12,960 these reversal interactions that we were 773 00:29:10,860 --> 00:29:15,480 mostly interested in and on the right 774 00:29:12,960 --> 00:29:17,400 you can see General vehicle interactions 775 00:29:15,480 --> 00:29:19,620 which also includes Vehicles holding or 776 00:29:17,400 --> 00:29:21,539 queuing up behind others 777 00:29:19,620 --> 00:29:24,120 now this approach really worked quite 778 00:29:21,539 --> 00:29:25,860 well and after this we conducted a 779 00:29:24,120 --> 00:29:28,080 second study 780 00:29:25,860 --> 00:29:30,720 um with a very different topic but the 781 00:29:28,080 --> 00:29:33,779 same methodology which basically looked 782 00:29:30,720 --> 00:29:35,460 at a mine site in wa and the road that 783 00:29:33,779 --> 00:29:37,020 is required to access it and this is a 784 00:29:35,460 --> 00:29:40,020 dirt road and the biggest problem there 785 00:29:37,020 --> 00:29:42,120 is dust development so we basically use 786 00:29:40,020 --> 00:29:44,760 this methodology to simulate different 787 00:29:42,120 --> 00:29:47,340 operating scenarios water trucks Convoy 788 00:29:44,760 --> 00:29:50,279 scenarios to get workers in and out and 789 00:29:47,340 --> 00:29:53,100 heavy Vehicles traveling up and down 790 00:29:50,279 --> 00:29:55,260 this road in machinery 791 00:29:53,100 --> 00:29:57,480 um yeah so a very useful little 792 00:29:55,260 --> 00:29:59,580 methodology and as you can see the 793 00:29:57,480 --> 00:30:02,220 special context is really important and 794 00:29:59,580 --> 00:30:04,380 useful to show your results 795 00:30:02,220 --> 00:30:07,279 um that's it from us if there's still 796 00:30:04,380 --> 00:30:11,120 time I think happy to take questions 797 00:30:07,279 --> 00:30:11,120 otherwise thanks for listening 798 00:30:14,940 --> 00:30:18,840 thank you sir 799 00:30:16,159 --> 00:30:20,880 thank you so much for that talk um we 800 00:30:18,840 --> 00:30:22,740 are running a bit late and so um you did 801 00:30:20,880 --> 00:30:24,539 a wonderful job with a few minutes extra 802 00:30:22,740 --> 00:30:25,919 but we might give that back to folks too 803 00:30:24,539 --> 00:30:29,039 because we're going to try and stick to 804 00:30:25,919 --> 00:30:30,840 the 11 o'clock schedule so uh we'll jump 805 00:30:29,039 --> 00:30:32,460 into saying thank you so much Christine 806 00:30:30,840 --> 00:30:36,860 and and long 807 00:30:32,460 --> 00:30:36,860 um and we have some some gifts for you 808 00:30:38,220 --> 00:30:43,220 foreign 809 00:30:40,100 --> 00:30:43,220 thank you 810 00:30:45,000 --> 00:30:47,059 um