python - Speed up Numpy Meshgrid Command -
i generating meshgrid numpy , it's taking lot of memory , quite bit of time well.
xi, yi = np.meshgrid(xi, yi)
i generating meshgrid same resolution underlying sitemap image, 3000px dimensions. uses several gigs of memory , takes 10-15 seconds or more while when it's writing page file.
my question is; can speed without upgrading server? here full copy of application source code.
def generatecontours(date_collected, substance_name, well_arr, site_id, sitemap_id, image, title_wildcard='', label_over_well=false, crop_contours=false, groundwater_contours=false, flow_lines=false, site_image_alpha=1, status_token=""): #create empty arrays fill up! x_values = [] y_values = [] z_values = [] #iterate on wells , fill arrays data in well_arr: x_values.append(well['xpos']) y_values.append(well['ypos']) z_values.append(well['value']) #initialize numpy array required interpolation functions x = np.array(x_values, dtype=np.float) y = np.array(y_values, dtype=np.float) z = np.array(z_values, dtype=np.float) #create list of x, y coordinate tuples points = zip(x, y) #create grid on interpolate data start_time = time.time() xi, yi = np.linspace(0, image['width'], image['width']), np.linspace(0, image['height'], image['height']) xi, yi = np.meshgrid(xi, yi) #interpolate data matlab griddata function (http://matplotlib.org/api/mlab_api.html#matplotlib.mlab.griddata) zi = griddata(x, y, z, xi, yi, interp='nn') #create matplotlib figure , adjust width , heights output contours resolution close original sitemap fig = plt.figure(figsize=(image['width']/72, image['height']/72)) #create single subplot, takes on whole figure if 1 specified ax = fig.add_subplot(111, frameon=false, xticks=[], yticks=[]) #read database image , save temporary variable im = image.open(image['tmpfile']) #place sitemap image on top of figure ax.imshow(im, origin='upper', alpha=site_image_alpha) #figure out linewidth if image['width'] > 2000: linewidth = 3 else: linewidth = 2 #create contours (options here http://cl.ly/2x0c311v2y01) kwargs = {} if groundwater_contours: kwargs['colors'] = 'b' cs = plt.contour(xi, yi, zi, linewidths=linewidth, **kwargs) key, value in enumerate(cs.levels): if value == 0: cs.collections[key].remove() #add streamplot if flow_lines: dy, dx = np.gradient(zi) plt.streamplot(xi, yi, dx, dy, color='c', density=1, arrowsize=3, arrowstyle='<-') #add labels locations label_kwargs = {} if label_over_well true: label_kwargs['manual'] = points plt.clabel(cs, cs.levels[1::1], inline=5, fontsize=math.floor(image['width']/100), fmt="%.1f", **label_kwargs) #add scatterplot show data read scatter_size = math.floor(image['width']/20) plt.scatter(x, y, s=scatter_size, c='k', facecolors='none', marker=(5, 1)) try: site_name = db_session.query(sites).filter_by(site_id=site_id).first().title except: site_name = "site map #%i" % site_id sitemap = sitemaps.query.get(sitemap_id) if sitemap.title != 'sitemap': sitemap_wildcard = " - " + sitemap.title else: sitemap_wildcard = "" if title_wildcard != '': filename_wildcard = "-" + slugify(title_wildcard) title_wildcard = " - " + title_wildcard else: filename_wildcard = "" title_wildcard = "" #add descriptive title top of contours title_font_size = math.floor(image['width']/72) plt.title(parsedate(date_collected) + " - " + site_name + " " + substance_name + " contour" + sitemap_wildcard + title_wildcard, fontsize=title_font_size) #generate unique filename , save temp directory filename = slugify(site_name) + str(int(time.time())) + filename_wildcard + ".pdf" temp_dir = tempfile.gettempdir() tempfileobj = temp_dir + "/" + filename savefig(tempfileobj) # bbox_inches='tight' tightens white border #clears matplotlib memory clf() #send temporary file user resp = make_response(send_file(tempfileobj, mimetype='application/pdf', as_attachment=true, attachment_filename=filename)) #set users status token javascript workaround check if file done being generated resp.set_cookie('status_token', status_token) return resp
if meshgrid
what's slowing down, don't call it... according griddata
docs:
xi , yi must describe regular grid, can either 1d or 2d, must monotonically increasing.
so call griddata
should work same if skip call meshgrid
, do:
xi = np.linspace(0, image['width'], image['width']) yi = np.linspace(0, image['height'], image['height']) zi = griddata(x, y, z, xi, yi, interp='nn')
this said, if x
, y
vectors large, actual interpolation, i.e. call griddata
going take quite time, delaunay triangulation computationally intensive operation. sure performanc issues coming meshgrid
, not griddata
?
Comments
Post a Comment