I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Check several frames average  intensity
Can you write Python code for it?

img_choice_N = 3
img_samp_index = random.sample( range(len(imgs)), img_choice_N)
avg_img =  get_avg_img( imgsa, img_samp_index, plot_ = False, uid =uidstr)
if avg_img.max() == 0:
    print('There are no photons recorded for this uid: %s'%uid)
    print('The data analysis should be terminated! Please try another uid.')

#show_img( imgsa[1000],  vmin=.1, vmax= 1e1, logs=True, aspect=1,
#         image_name= uidstr + '_img_avg',  save=True, path=data_dir,  cmap = cmap_albula )
print(center[::-1])

show_img( imgsa[ 5],  vmin = -1, vmax = 20, logs=False, aspect=1, #save_format='tif',
         image_name= uidstr + '_img_avg',  save=True, path=data_dir, cmap=cmap_albula,center=center[::-1])
# select subregion, hard coded center beam location
#show_img( imgsa[180+40*3/0.05][110:110+840*2, 370:370+840*2],  vmin = 0.01, vmax = 20, logs=False, aspect=1, #save_format='tif',
#         image_name= uidstr + '_img_avg',  save=True, path=data_dir, cmap=cmap_albula,center=[845,839])


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Compress Data

Generate a compressed data with filename
Replace old mask with a new mask with removed hot pixels
Do average image
Do each image sum
Find badframe_list for where image sum above bad_pixel_threshold
Check shutter open frame to get good time series
Can you write Python code for it?

compress=True
photon_occ = len( np.where(avg_img)[0] ) / ( imgsa[0].size)
#compress =  photon_occ < .4  #if the photon ocupation < 0.5, do compress
print ("The non-zeros photon occupation is %s."%( photon_occ))
print("Will " + 'Always ' + ['NOT', 'DO'][compress]  + " apply compress process.")

if  md['detector'] =='eiger4m_single_image' or md['detector'] == 'image':
    good_start =    5  #make the good_start at least 0
elif md['detector'] =='eiger500K_single_image':
    good_start = 100  #5  #make the good_start at least 0

elif  md['detector'] =='eiger1m_single_image' or md['detector'] == 'image':
    good_start =    5

bin_frame =  False # True  #generally make bin_frame as False
if bin_frame:
    bin_frame_number=4
    acquisition_period = md['acquire period']
    timeperframe = acquisition_period * bin_frame_number
else:
    bin_frame_number =1

force_compress = False
#force_compress = True

import time
t0= time.time()

if not use_local_disk:
    cmp_path = '/nsls2/xf11id1/analysis/Compressed_Data'
else:
    cmp_path = '/tmp_data/compressed'

cmp_path = '/nsls2/xf11id1/analysis/Compressed_Data'
if bin_frame_number==1:
    cmp_file = '/uid_%s.cmp'%md['uid']
else:
    cmp_file = '/uid_%s_bined--%s.cmp'%(md['uid'],bin_frame_number)

filename = cmp_path + cmp_file
mask2, avg_img, imgsum, bad_frame_list = compress_eigerdata(imgs, mask, md, filename,
         force_compress= force_compress,  para_compress= para_compress,  bad_pixel_threshold = 1e14,
                                    reverse=reverse, rot90=rot90,
                        bins=bin_frame_number, num_sub= 100, num_max_para_process= 500, with_pickle=True,
                        direct_load_data =use_local_disk, data_path = data_fullpath, )

min_inten = 10
good_start = max(good_start, np.where( np.array(imgsum) > min_inten )[0][0] )
print ('The good_start frame number is: %s '%good_start)
FD =...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Creat new mask by masking the bad pixels and get new avg_img
Can you write Python code for it?

imgsum_y = imgsum[ np.array( [i for i in np.arange( len(imgsum)) if i not in bad_frame_list])]
imgsum_x = np.arange( len( imgsum_y))
save_lists(  [imgsum_x, imgsum_y], label=['Frame', 'Total_Intensity'],
           filename=uidstr + '_img_sum_t', path= data_dir  )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Do a waterfall analysis
Can you write Python code for it?

qth_interest = 2 #the second ring. #qth_interest starting from 1
if scat_geometry =='saxs' or scat_geometry =='gi_waxs':
    if run_waterfall:
        wat = cal_waterfallc( FD, roi_mask, qindex= qth_interest, save =True, path=data_dir, uid=uidstr)
        plot_waterfallc( wat, qth_interest, aspect= None, vmin=1e-1, vmax= wat.max(), uid=uidstr, save =True,
                        path=data_dir, beg= FD.beg, cmap = cmap_vge )

q_mask_name

ring_avg = None
if run_t_ROI_Inten:
    times_roi, mean_int_sets = cal_each_ring_mean_intensityc(FD, roi_mask, timeperframe = None, multi_cor=True  )
    plot_each_ring_mean_intensityc( times_roi, mean_int_sets,  uid = uidstr, save=True, path=data_dir )
    roi_avg = np.average( mean_int_sets, axis=0)


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Do historam
Can you write Python code for it?

if run_xsvs:
    if roi_avg is  None:
        times_roi, mean_int_sets = cal_each_ring_mean_intensityc(FD, roi_mask, timeperframe = None,  )
        roi_avg = np.average( mean_int_sets, axis=0)

    t0=time.time()
    spec_bins, spec_his, spec_std, spec_sum  =  xsvsp( FD, np.int_(roi_mask), norm=None,
                max_cts=int(max_cts+2),  bad_images=bad_frame_list, only_two_levels=True )
    spec_kmean =  np.array(  [roi_avg * 2**j for j in  range( spec_his.shape[0] )] )
    run_time(t0)
    spec_pds =  save_bin_his_std( spec_bins, spec_his, spec_std, filename=uid_+'_spec_res.csv', path=data_dir )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Don't Change the lines below here

get metadata
Can you write Python code for it?

md = get_meta_data( uid )
md_blue = md.copy()
#md_blue

#md_blue['detectors'][0]
#if md_blue['OAV_mode'] != 'none':
#    cx , cy = md_blue[md_blue['detectors'][0]+'_beam_center_x'], md_blue[md_blue['detectors'][0]+'_beam_center_x']
#else:
#    cx , cy = md_blue['beam_center_x'], md_blue['beam_center_y']
#print(cx,cy)

detectors = sorted(get_detectors(db[uid]))
print('The detectors are:%s'%detectors)
if len(detectors) >1:
    md['detector'] = detectors[1]
    print( md['detector'])

if md['detector'] =='eiger4m_single_image' or md['detector'] == 'image':
    reverse= True
    rot90= False
elif md['detector'] =='eiger500K_single_image':
    reverse= True
    rot90=True
elif md['detector'] =='eiger1m_single_image':
    reverse= True
    rot90=False
print('Image reverse: %s\nImage rotate 90: %s'%(reverse, rot90))

try:
    cx , cy = md_blue['beam_center_x'], md_blue['beam_center_y']
    print(cx,cy)
except:
    print('Will find cx,cy later.')


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Dynamic Analysis for gi_saxs
Can you write Python code for it?

if scat_geometry =='gi_saxs':
    show_qzr_roi( avg_img, roi_mask, inc_x0, ticks[:4], alpha=0.5, save=True, path=data_dir, uid=uidstr )
    ## Get 1D Curve (Q||-intensity¶)
    qr_1d_pds = cal_1d_qr( avg_img, Qr, Qz, qr_map, qz_map, inc_x0= None, mask=mask, setup_pargs=setup_pargs )
    plot_qr_1d_with_ROI( qr_1d_pds, qr_center=np.unique( np.array(list( qval_dict.values() ) )[:,0] ),
                    loglog=True, save=True, uid=uidstr, path = data_dir)


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Export Results to a HDF5 File
Can you write Python code for it?

md['mask_file']= mask_path + mask_name
md['roi_mask_file']= fp
md['mask'] = mask
#md['NOTEBOOK_FULL_PATH'] =  data_dir + get_current_pipeline_fullpath(NFP).split('/')[-1]
md['good_start'] = good_start
md['bad_frame_list'] = bad_frame_list
md['avg_img'] = avg_img
md['roi_mask'] = roi_mask
md['setup_pargs'] = setup_pargs
if scat_geometry == 'gi_saxs':
    md['Qr'] = Qr
    md['Qz'] = Qz
    md['qval_dict'] = qval_dict
    md['beam_center_x'] =  inc_x0
    md['beam_center_y']=   inc_y0
    md['beam_refl_center_x'] = refl_x0
    md['beam_refl_center_y'] = refl_y0

elif scat_geometry == 'gi_waxs':
    md['beam_center_x'] =  center[1]
    md['beam_center_y']=  center[0]
else:
    md['qr']= qr
    #md['qr_edge'] = qr_edge
    md['qval_dict'] = qval_dict
    md['beam_center_x'] =  center[1]
    md['beam_center_y']=  center[0]

md['beg'] = FD.beg
md['end'] = FD.end
md['t_print0'] = t_print0
md['qth_interest'] = qth_interest
md['metadata_file'] = data_dir + 'uid=%s_md.pkl'%uid

psave_obj(  md, data_dir + 'uid=%s_md.pkl'%uid ) #save the setup parameters
save_dict_csv( md,  data_dir + 'uid=%s_md.csv'%uid, 'w')

Exdt = {}
if scat_geometry == 'gi_saxs':
    for k,v in zip( ['md', 'roi_mask','qval_dict','avg_img','mask','pixel_mask', 'imgsum', 'bad_frame_list', 'qr_1d_pds'],
                [md,    roi_mask, qval_dict, avg_img,mask,pixel_mask, imgsum, bad_frame_list, qr_1d_pds] ):
        Exdt[ k ] = v
elif scat_geometry == 'saxs':
    for k,v in zip( ['md', 'q_saxs', 'iq_saxs','iqst','qt','roi_mask','qval_dict','avg_img','mask','pixel_mask', 'imgsum', 'bad_frame_list'],
                [md, q_saxs, iq_saxs, iqst, qt,roi_mask, qval_dict, avg_img,mask,pixel_mask, imgsum, bad_frame_list] ):
        Exdt[ k ] = v
elif scat_geometry == 'gi_waxs':
    for k,v in zip( ['md', 'roi_mask','qval_dict','avg_img','mask','pixel_mask', 'imgsum', 'bad_frame_list'],
                [md,       roi_mask, qval_dict, avg_img,mask,pixel_mask, imgsum, bad_frame_list] ):
        Exdt[ k ] = v

if...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Fit g2
Can you write Python code for it?

if run_one_time:
    g2_fit_result, taus_fit, g2_fit = get_g2_fit_general( g2,  taus,
                function = fit_g2_func,  vlim=[0.95, 1.05], fit_range= None,
            fit_variables={'baseline':False, 'beta': True, 'alpha':True,'relaxation_rate':True,},
            guess_values={'baseline':1.0,'beta': 0.03,'alpha':1.0,'relaxation_rate':0.0005},
            guess_limits = dict( baseline =[.9, 1.3], alpha=[0, 2],
                        beta = [0, 1], relaxation_rate= [1e-7, 1000]) ,)
    g2_fit_paras = save_g2_fit_para_tocsv(g2_fit_result,  filename= uid_ +'_'+q_mask_name +'_g2_fit_paras.csv', path=data_dir )

scat_geometry_

if run_one_time:
    if cal_g2_error:
        g2_fit_err = np.zeros_like(g2_fit)
        plot_g2_general( g2_dict={1:g2, 2:g2_fit}, taus_dict={1:taus, 2:taus_fit},
                    vlim=[0.95, 1.05], g2_err_dict= {1:g2_err, 2: g2_fit_err},
                qval_dict = dict(itertools.islice(qval_dict.items(),g2.shape[1])), fit_res= g2_fit_result,  geometry= scat_geometry_,filename= uid_+'_g2',
        path= data_dir, function= fit_g2_func,  ylabel='g2', append_name=  '_fit')
    else:
        plot_g2_general( g2_dict={1:g2, 2:g2_fit}, taus_dict={1:taus, 2:taus_fit}, vlim=[0.95, 1.05],
                qval_dict = dict(itertools.islice(qval_dict.items(),g2.shape[1])), fit_res= g2_fit_result,  geometry= scat_geometry_,filename= uid_+'_g2',
        path= data_dir, function= fit_g2_func,  ylabel='g2', append_name=  '_fit')

if run_one_time:
    if True:
        fs, fe = 0, 8
        #fs,fe=0, 6
        qval_dict_ = {k:qval_dict[k] for k in list(qval_dict.keys())[fs:fe]  }
        D0, qrate_fit_res = get_q_rate_fit_general(  qval_dict_, g2_fit_paras['relaxation_rate'][fs:fe],
                                                   geometry=  scat_geometry_ )
        plot_q_rate_fit_general( qval_dict_, g2_fit_paras['relaxation_rate'][fs:fe],  qrate_fit_res,
                                geometry= scat_geometry_,uid=uid_  , path=...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Four Time Correlation
Can you write Python code for it?

if run_four_time:
    t0=time.time()
    g4 = get_four_time_from_two_time(g12b, g2=g2b)[:int(max_taus)]
    run_time(t0)

if run_four_time:
    taus4 = np.arange( g4.shape[0])*timeperframe
    g4_pds = save_g2_general( g4, taus=taus4, qr=np.array( list( qval_dict.values() ) )[:,0],
                             qz=None, uid=uid_ +'_g4.csv', path= data_dir, return_res=True )

if run_four_time:
    plot_g2_general( g2_dict={1:g4}, taus_dict={1:taus4},vlim=[0.95, 1.05], qval_dict=qval_dict, fit_res= None,
                geometry=scat_geometry_,filename=uid_+'_g4',path= data_dir,   ylabel='g4')


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Get Dynamic Mask (currently designed for 500K)
Can you write Python code for it?

#%run /home/yuzhang/pyCHX_link/pyCHX/chx_packages.py

if  md['detector'] =='eiger4m_single_image' or md['detector'] == 'image':
    pass
elif md['detector'] =='eiger500K_single_image':
    #if md['cam_acquire_period'] <= 0.00015:  #will check this logic
    if imgs[0].dtype == 'uint16':
        print('Create dynamic mask for 500K due to 9K data acquistion!!!')
        bdp = find_bad_pixels_FD( bad_frame_list, FD, img_shape = avg_img.shape, threshold=20 )
        mask = mask_copy2.copy()
        mask *=bdp
        mask_copy = mask.copy()
        show_img(  mask, image_name='New Mask_uid=%s'%uid )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Get bad frame list by a polynominal fit
Can you write Python code for it?

good_end= None # 2000
if good_end is not None:
    FD = Multifile(filename, good_start, min( len(imgs)//bin_frame_number, good_end) )
    uid_ = uidstr + '_fra_%s_%s'%(FD.beg, FD.end)
    print( uid_ )

re_define_good_start =False
if re_define_good_start:
    good_start = 180
    #good_end = 19700
    good_end = len(imgs)
    FD = Multifile(filename, good_start, good_end)
    uid_ = uidstr + '_fra_%s_%s'%(FD.beg, FD.end)
    print( FD.beg, FD.end)

bad_frame_list =  get_bad_frame_list( imgsum, fit='both',  plot=True,polyfit_order = 30,
                        scale= 3.5,  good_start = good_start, good_end=good_end, uid= uidstr, path=data_dir)

print( 'The bad frame list length is: %s'%len(bad_frame_list) )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Get contrast
Can you write Python code for it?

if run_xsvs:
    contrast_factorL = get_contrast( ML_val)
    spec_km_pds = save_KM(  spec_kmean, KL_val, ML_val, qs=qr, level_time=times_xsvs, uid=uid_, path = data_dir )
    #spec_km_pds


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
GiSAXS Scattering Geometry
Can you write Python code for it?

if scat_geometry =='gi_saxs':
    plot_qzr_map(  qr_map, qz_map, inc_x0, ticks = ticks, data= avg_img, uid= uidstr, path = data_dir   )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Load Chip mask depeding on detector
Can you write Python code for it?

if md['detector'] =='eiger1m_single_image':
    Chip_Mask=np.load( '/XF11ID/analysis/2017_1/masks/Eiger1M_Chip_Mask.npy')
elif md['detector'] =='eiger4m_single_image' or md['detector'] == 'image':
    Chip_Mask= np.array(np.load( '/XF11ID/analysis/2017_1/masks/Eiger4M_chip_mask.npy'), dtype=bool)
    BadPix =     np.load('/XF11ID/analysis/2018_1/BadPix_4M.npy'  )
    Chip_Mask.ravel()[BadPix] = 0
elif md['detector'] =='eiger500K_single_image':
    #print('here')
    Chip_Mask=  np.load( '/XF11ID/analysis/2017_1/masks/Eiger500K_Chip_Mask.npy')  #to be defined the chip mask
    Chip_Mask = np.rot90(Chip_Mask)
    pixel_mask = np.rot90(  1- np.int_( np.array( imgs.md['pixel_mask'], dtype= bool))   )

else:
    Chip_Mask = 1
#show_img(Chip_Mask)
print(Chip_Mask.shape, pixel_mask.shape)

use_local_disk = True
import shutil,glob

save_oavs = False
if len(detectors)==2:
    if '_image' in md['detector']:
        pref = md['detector'][:-5]
    else:
        pref=md['detector']
    for k in [ 'beam_center_x', 'beam_center_y','cam_acquire_time','cam_acquire_period','cam_num_images',
             'wavelength', 'det_distance', 'photon_energy']:
        md[k] =  md[ pref + '%s'%k]

    if 'OAV_image' in detectors:
        try:
            #tifs =  list( db[uid].data(  'OAV_image') )[0]
            #print(len(tifs))
            save_oavs_tifs(  uid, data_dir )
            save_oavs = True
            ## show all images
            #fig, ax = show_tif_series( tifs, Nx = None, vmin=1.0, vmax=20, logs=False,
            #                    cmap= cm.gray, figsize=[4,6] )
            ##show one image
            #show_img(tifs[0],cmap= cm.gray,)
        except:
            pass

print_dict( md,  ['suid', 'number of images', 'uid', 'scan_id', 'start_time', 'stop_time', 'sample', 'Measurement',
                  'acquire period', 'exposure time',
         'det_distance', 'beam_center_x', 'beam_center_y', ] )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Load ROI defined by "XPCS_Setup" Pipeline
Define data analysis type
Can you write Python code for it?

if analysis_type_auto:#if True, will take "analysis type" option from data acquisition func series
    try:
        qphi_analysis_ = md['analysis'] #if True, will do q-phi (anisotropic analysis for transmission saxs)
        print(md['analysis'])
        if qphi_analysis_ == 'iso':
            qphi_analysis = False
        elif qphi_analysis_ == '':
            qphi_analysis = False
        else:
            qphi_analysis = True
    except:
        print('There is no analysis in metadata.')

print('Will %s qphis analysis.'%['NOT DO','DO'][qphi_analysis])

if scat_geometry != 'saxs':qphi_analysis = False;scat_geometry_ = scat_geometry
else:scat_geometry_ = ['','ang_'][qphi_analysis]+ scat_geometry
if scat_geometry != 'gi_saxs':run_profile_plot = False
print(scat_geometry_)

#isotropic_Q_mask

scat_geometry


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Load ROI mask depending on data analysis type
Can you write Python code for it?

##For SAXS
roi_path = '/XF11ID/analysis/2019_1/masks/'
roi_date = 'Feb6'

if scat_geometry =='saxs':
    if qphi_analysis ==  False:
        if isotropic_Q_mask == 'normal':
            #print('Here')
            q_mask_name='rings'
            if md['detector'] =='eiger4m_single_image' or md['detector'] == 'image': #for 4M
                fp = roi_path + 'roi_mask_%s_4M_norm.pkl'%roi_date

            elif md['detector'] =='eiger500K_single_image': #for 500K
                fp = roi_path + 'roi_mask_%s_500K_norm.pkl'%roi_date

        elif isotropic_Q_mask == 'wide':
            q_mask_name='wide_rings'
            if md['detector'] =='eiger4m_single_image' or md['detector'] == 'image': #for 4M
                fp = roi_path + 'roi_mask_%s_4M_wide.pkl'%roi_date
            elif md['detector'] =='eiger500K_single_image': #for 500K
                fp = roi_path + 'roi_mask_%s_500K_wide.pkl'%roi_date

    elif qphi_analysis:
        if phi_Q_mask =='phi_4x_20deg':
            q_mask_name='phi_4x_20deg'
            if md['detector'] =='eiger4m_single_image' or md['detector'] == 'image': #for 4M
                fp = roi_path + 'roi_mask_%s_4M_phi_4x_20deg.pkl'%roi_date
            elif md['detector'] =='eiger500K_single_image': #for 500K
                fp = roi_path + 'roi_mask_%s_500K_phi_4x_20deg.pkl'%roi_date

    #fp = 'XXXXXXX.pkl'
    roi_mask,qval_dict = cpk.load( open(fp, 'rb' )  )  #for load the saved roi data
    #print(fp)

## Gi_SAXS
elif scat_geometry =='gi_saxs':
    # dynamics mask
    fp = '/XF11ID/analysis/2018_2/masks/uid=460a2a3a_roi_mask.pkl'
    roi_mask,qval_dict = cpk.load( open(fp, 'rb' )  )  #for load the saved roi data
    print('The dynamic mask is: %s.'%fp)
    # static mask
    fp = '/XF11ID/analysis/2018_2/masks/uid=460a2a3a_roi_masks.pkl'
    roi_masks,qval_dicts = cpk.load( open(fp, 'rb' )  )  #for...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Make a directory for saving results
Can you write Python code for it?

CYCLE= '2019_1'  #change clycle here
path = '/XF11ID/analysis/%s/masks/'%CYCLE
username =  getpass.getuser()
username = 'commisionning'
username = 'petrash'

data_dir0  = create_user_folder(CYCLE, username)
print( data_dir0 )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
One time Correlation
Note : Enter the number of buffers for Muliti tau one time correlation
number of buffers has to be even. More details in https://github.com/scikit-beam/scikit-beam/blob/master/skbeam/core/correlation.py
if define another good_series
Can you write Python code for it?

define_good_series = False
#define_good_series = True
if define_good_series:
    good_start = 200
    FD = Multifile(filename, beg = good_start, end = 600) #end=1000)
    uid_ = uidstr + '_fra_%s_%s'%(FD.beg, FD.end)
    print( uid_ )

if use_sqnorm:#for transmision SAXS
    norm = get_pixelist_interp_iq( qp_saxs, iq_saxs, roi_mask, center)
    print('Using circular average in the normalization of G2 for SAXS scattering.')
elif use_SG:#for Gi-SAXS or WAXS
    avg_imgf = sgolay2d( avg_img, window_size= 11, order= 5) * mask
    norm=np.ravel(avg_imgf)[pixelist]
    print('Using smoothed image by SavitzkyGolay filter in the normalization of G2.')
else:
    norm= None
    print('Using simple (average) normalization of G2.')

if use_imgsum_norm:
    imgsum_ = imgsum
    print('Using frame total intensity for intensity normalization in g2 calculation.')
else:
    imgsum_ = None
import time

if run_one_time:
    t0 = time.time()
    if cal_g2_error:
        g2,lag_steps,g2_err = cal_g2p(FD,roi_mask,bad_frame_list,good_start, num_buf = 8,
                            num_lev= None,imgsum= imgsum_, norm=norm, cal_error= True )
    else:
        g2,lag_steps    =     cal_g2p(FD,roi_mask,bad_frame_list,good_start, num_buf = 8,
                            num_lev= None,imgsum= imgsum_, norm=norm, cal_error= False )

    run_time(t0)

#g2_err.shape, g2.shape

lag_steps = lag_steps[:g2.shape[0]]
g2.shape[1]

if run_one_time:

    taus = lag_steps * timeperframe
    try:
        g2_pds = save_g2_general( g2, taus=taus,qr= np.array( list( qval_dict.values() ) )[:g2.shape[1],0],
                                            qz = np.array( list( qval_dict.values() ) )[:g2.shape[1],1],
                             uid=uid_+'_g2.csv', path= data_dir, return_res=True )
    except:
        g2_pds = save_g2_general( g2, taus=taus,qr= np.array( list( qval_dict.values() ) )[:g2.shape[1],0],
                            ...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Overwrite Some Metadata if Wrong Input
Define incident beam center (also define reflection beam center for gisaxs)
Can you write Python code for it?

if scat_geometry =='gi_saxs':
    inc_x0 =  md['beam_center_x']
    inc_y0 =  imgs[0].shape[0] - md['beam_center_y']
    refl_x0 =     md['beam_center_x']
    refl_y0 =     1000     #imgs[0].shape[0] -  1758

    print( "inc_x0, inc_y0, ref_x0,ref_y0 are: %s %s %s %s."%(inc_x0, inc_y0, refl_x0, refl_y0) )
else:
    if md['detector'] =='eiger4m_single_image' or md['detector'] == 'image' or md['detector']=='eiger1m_single_image':
        inc_x0 =  imgs[0].shape[0] - md['beam_center_y']
        inc_y0=   md['beam_center_x']
    elif md['detector'] =='eiger500K_single_image':
        inc_y0 =  imgs[0].shape[1] - md['beam_center_y']
        inc_x0 =   imgs[0].shape[0] - md['beam_center_x']

    print(inc_x0, inc_y0)
    ###for this particular uid, manually give x0/y0
    #inc_x0 = 1041
    #inc_y0 = 1085

dpix, lambda_, Ldet,  exposuretime, timeperframe, center = check_lost_metadata(
    md, Nimg, inc_x0 = inc_x0, inc_y0=   inc_y0, pixelsize = 7.5*10*(-5) )
if scat_geometry =='gi_saxs':center=center[::-1]
setup_pargs=dict(uid=uidstr, dpix= dpix, Ldet=Ldet, lambda_= lambda_, exposuretime=exposuretime,
        timeperframe=timeperframe, center=center, path= data_dir)
print_dict( setup_pargs )

setup_pargs


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Plot fit results
Can you write Python code for it?

if run_xsvs:
    qr = [qval_dict[k][0] for k in list(qval_dict.keys()) ]
    plot_xsvs_fit(  spec_his, ML_val, KL_val, K_mean = spec_kmean, spec_std=spec_std,
                  xlim = [0,10], vlim =[.9, 1.1],
        uid=uid_, qth= qth_interest, logy= True, times= times_xsvs, q_ring_center=qr, path=data_dir)

    plot_xsvs_fit(  spec_his, ML_val, KL_val, K_mean = spec_kmean, spec_std = spec_std,
                  xlim = [0,15], vlim =[.9, 1.1],
        uid=uid_, qth= None, logy= True, times= times_xsvs, q_ring_center=qr, path=data_dir )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Plot time~ total intensity of each frame
Can you write Python code for it?

plot1D( y = imgsum_y, title = uidstr + '_img_sum_t', xlabel='Frame', c='b',
       ylabel='Total_Intensity', legend='imgsum', save=True, path=data_dir)


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Run Dose dependent analysis
Can you write Python code for it?

#run_dose = True

if run_dose:
    get_two_time_mulit_uids( [uid], roi_mask,  norm= norm,  bin_frame_number=1,
                        path= data_dir0, force_generate=False, compress_path =  cmp_path + '/' )

try:
    print( md['transmission'] )
except:
    md['transmission'] =1

exposuretime

if run_dose:
    N = len(imgs)
    print(N)
    #exposure_dose = md['transmission'] * exposuretime* np.int_([     N/16, N/8, N/4 ,N/2, 3*N/4, N*0.99 ])
    exposure_dose = md['transmission'] * exposuretime* np.int_([      N/8, N/4 ,N/2, 3*N/4, N*0.99 ])

    print( exposure_dose )

if run_dose:
    taus_uids, g2_uids = get_series_one_time_mulit_uids( [ uid ],  qval_dict, good_start=good_start,
                    path= data_dir0, exposure_dose = exposure_dose,  num_bufs =8, save_g2= False,
                                                   dead_time = 0, trans = [ md['transmission'] ] )

if run_dose:
    plot_dose_g2( taus_uids, g2_uids, ylim=[1.0, 1.2], vshift= 0.00,
                 qval_dict = qval_dict, fit_res= None,  geometry= scat_geometry_,
                 filename= '%s_dose_analysis'%uid_,
                path= data_dir, function= None,  ylabel='g2_Dose', g2_labels= None, append_name=  '' )

if run_dose:
    qth_interest = 1
    plot_dose_g2( taus_uids, g2_uids, qth_interest= qth_interest, ylim=[0.98, 1.2], vshift= 0.00,
                 qval_dict = qval_dict, fit_res= None,  geometry= scat_geometry_,
                 filename= '%s_dose_analysis'%uidstr,
                path= data_dir, function= None,  ylabel='g2_Dose', g2_labels= None, append_name=  '' )

0.33/0.00134


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Save the OVA image
Can you write Python code for it?

if save_oavs:
    os.environ['HTTPS_PROXY'] = 'https://proxy:8888'
    os.environ['no_proxy'] = 'cs.nsls2.local,localhost,127.0.0.1'
    update_olog_uid_with_file( uid[:6], text='Add OVA images',
                              filename= data_dir + 'uid=%s_OVA_images.png'%uid, append_name='_img' )
       # except:


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Save the current pipeline in Results folder
Can you write Python code for it?

#save_current_pipeline( NFP, data_dir)

#get_current_pipeline_fullpath(NFP)


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Speckle Visiblity
Can you write Python code for it?

#run_xsvs =True

if run_xsvs:
    max_cts = get_max_countc(FD, roi_mask )
    #max_cts = 15 #for eiger 500 K
    qind, pixelist = roi.extract_label_indices(   roi_mask  )
    noqs = len( np.unique(qind) )
    nopr = np.bincount(qind, minlength=(noqs+1))[1:]
    #time_steps = np.array( utils.geometric_series(2,   len(imgs)   ) )
    time_steps = [0,1]  #only run the first two levels
    num_times = len(time_steps)
    times_xsvs = exposuretime + (2**(  np.arange( len(time_steps) ) ) -1 ) * timeperframe
    print( 'The max counts are: %s'%max_cts )


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
Static Analysis
SAXS Scattering Geometry
Can you write Python code for it?

setup_pargs

#%run ~/pyCHX_link/pyCHX/chx_generic_functions.py
%run ~/pyCHX_link/pyCHX/XPCS_SAXS.py

if scat_geometry =='saxs':
    ## Get circular average| * Do plot and save q~iq
    mask = mask_copy.copy()
    hmask = create_hot_pixel_mask( avg_img, threshold = 1e8, center=center, center_radius= 10)

    qp_saxs, iq_saxs, q_saxs = get_circular_average( avg_img * Chip_Mask , mask * hmask, pargs=setup_pargs  )
    plot_circular_average( qp_saxs, iq_saxs, q_saxs,  pargs=setup_pargs,  show_pixel=True,
                      xlim=[qp_saxs.min(), qp_saxs.max()*1.0], ylim = [iq_saxs.min(), iq_saxs.max()*2] )
    mask =np.array( mask * hmask, dtype=bool)

if scat_geometry =='saxs':
    if run_fit_form:
        form_res = fit_form_factor( q_saxs,iq_saxs,  guess_values={'radius': 2500, 'sigma':0.05,
         'delta_rho':1E-10 },  fit_range=[0.0001, 0.015], fit_variables={'radius': T, 'sigma':T,
         'delta_rho':T},  res_pargs=setup_pargs, xlim=[0.0001, 0.015])

    qr = np.array( [qval_dict[k][0] for k in sorted( qval_dict.keys())] )
    if qphi_analysis ==  False:
        try:
            qr_cal, qr_wid = get_QrQw_From_RoiMask( roi_mask, setup_pargs  )
            print(len(qr))
            if (qr_cal - qr).sum() >=1e-3:
                print( 'The loaded ROI mask might not be applicable to this UID: %s.'%uid)
                print('Please check the loaded roi mask file.')
        except:
            print('Something is wrong with the roi-mask. Please check the loaded roi mask file.')

    show_ROI_on_image( avg_img*roi_mask, roi_mask, center, label_on = False, rwidth = 840, alpha=.9,
                 save=True, path=data_dir, uid=uidstr, vmin= 1e-3,
                 vmax= 1e-1, #np.max(avg_img),
                 aspect=1,
                 show_roi_edge=True,
                 show_ang_cor = True)
    plot_qIq_with_ROI( q_saxs, iq_saxs, np.unique(qr), logs=True, uid=uidstr,
                      xlim=[q_saxs.min(),...


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
The End!
Can you write Python code for it?


I am working on the file "2019_1/Template/XPCS_Single_2019_V2.ipynb".
The first task is:
get data
Can you write Python code for it?

imgs = load_data( uid, md['detector'], reverse= reverse, rot90=rot90  )
md.update( imgs.md );Nimg = len(imgs);
#md['beam_center_x'], md['beam_center_y']  = cx, cy
#if 'number of images'  not in list(md.keys()):
md['number of images']  = Nimg
pixel_mask =  1- np.int_( np.array( imgs.md['pixel_mask'], dtype= bool)  )
print( 'The data are: %s' %imgs )

#md['acquire period' ] = md['cam_acquire_period']
#md['exposure time'] =  md['cam_acquire_time']
mdn = md.copy()


I am working on the file "2019_RDM_course_nix.ipynb".
The first task is:
Now we are finally set up to put our data in our file!
Can you write Python code for it?

# First lets check how we can actually create a DataArray
help(block.create_data_array)

# Now we create the DataArray within the Block created above and with the data
# We also add the appropriate labels immediately.

da = block.create_data_array(name="data_regular", array_type="sine", data=voltage)
da.label = "voltage"
da.unit = "mV"

# Now we will also add the appropriate Dimension to this DataArray, so it can be correctly interpreted for
# later plotting. We will look into the different Dimensions in a second.

# Note that we always should add dimensions in the order x, y, z ... when thinking in plot terms
# This is necessary to later properly interpret data without knowing the actual structure of a DataArray.

# First we check how to properly create the Dimension we need
help(da.append_sampled_dimension)

# And lets add the Dimension of our X axis to our DataArray:
dim = da.append_sampled_dimension(sample_interval)
dim.label = "time"
dim.unit = "s"

# We also want to add a Dimension to our Y axis to make the DataArray consistent even if we do not add
# any additional annotations. We will see what a set dimension is later on
dim_set = da.append_set_dimension()


I am working on the file "2019_RDM_course_nix.ipynb".
The first task is:
Referencing a single point or region in a DataArray
To reference only a single point or region, we can use a NIX tag. The NIX tag is a simpler form of the MultiTag that we will cover in a moment.
Can you write Python code for it?

# We will create some more elaborate example data to make a point
# For this we need some equally elaborate code, which can be safely ignored.

# This code will create some mock membrane voltage traces for us

# ---- MOCK CODE AND DATA; the code can be safely ignored --------

class LIF(object):
    def __init__(self, stepsize=0.0001, offset=1.6, tau_m=0.025, tau_a=0.02, da=0.0, D=3.5):
        self.stepsize = stepsize  # simulation stepsize [s]
        self.offset = offset  # offset curent [nA]
        self.tau_m = tau_m  # membrane time_constant [s]
        self.tau_a = tau_a  # adaptation time_constant [s]
        self.da = da  # increment in adaptation current [nA]
        self.D = D  # noise intensity
        self.v_threshold = 1.0  # spiking threshold
        self.v_reset = 0.0  # reset voltage after spiking
        self.i_a = 0.0  # current adaptation current
        self.v = self.v_reset  # current membrane voltage
        self.t = 0.0  # current time [s]
        self.membrane_voltage = []
        self.spike_times = []

    def _reset(self):
        self.i_a = 0.0
        self.v = self.v_reset
        self.t = 0.0
        self.membrane_voltage = []
        self.spike_times = []

    def _lif(self, stimulus, noise):
        """
        euler solution of the membrane equation with adaptation current and noise
        """
        self.i_a -= self.i_a - self.stepsize/self.tau_a * (self.i_a)
        self.v += self.stepsize * ( -self.v + stimulus + noise + self.offset - self.i_a)/self.tau_m;
        self.membrane_voltage.append(self.v)

    def _next(self, stimulus):
        """
        working horse which delegates to the euler and gets the spike times
        """
        noise = self.D * (float(np.random.randn() % 10000) - 5000.0)/10000
        self._lif(stimulus, noise)
        self.t += self.stepsize
        if self.v > self.v_threshold and len(self.membrane_voltage) > 1:
            self.v = self.v_reset
            self.membrane_voltage[len(self.membrane_voltage)-1] = 2.0
           ...


I am working on the file "2019_RDM_course_nix.ipynb".
The first task is:
This is perfect data, we would like to keep it and store it in a file. So lets persist this wonderful data in a NIX file.
The DataArray
The DataArray is the most central entity of the NIX data model. As almost all other NIX-entities it requires a name and a type. Both are not restricted but names must be unique inside a Block. type information can be used to introduce semantic meaning and domain-specificity. Upon creation, a unique ID will be assigned to the DataArray.
The DataArray stores the actual data together with label and unit. In addition, the DataArray needs a dimension descriptor for each dimension. The following snippet shows how to create a DataArray and store data in it.
Can you write Python code for it?

import nixio

# First create a file we'll use to work with

# Files can be opened in FileMode "ReadOnly", "ReadWrite" and "Overwrite"
# ReadOnly ... Opens an existing file for reading
# ReadWrite ... Opens an existing file for editing or creates a new file
# Overwrite ... Truncates and opens an existing file or creates a new file
f = nixio.File.open('Tutorial.nix', nixio.FileMode.Overwrite)

# Please note, that nix works on an open file and reads and writes directly from and to this file.
# Always close the file using 'f.close()' when you are done.


I am working on the file "2019_RDM_course_nix.ipynb".
The first task is:
We can extract information from multiple steps of analysis and are able to plot data and analyses data without having to know or directly access the DataArrays that contain the acutal data.
Data and data annotation in the same file
NIX does not only allow to save initial data and analysed data within in the same file. It also allows to create structured annotations of the experiments that were conducted and connects this information directly to the data.
Metadata in NIX files is stored in the odML format and is saved in parallel to the actual "DataTree" in a "MetadataTree" but can easily be connected to Data in the DataTree.
odML is a hierarchically structured data format, that provides grouping in nestable 'Sections' and stores information in 'Property'-'Value' pairs.
Can you write Python code for it?

# Let us annotate the DataArray in our last example.

# As we can see, we have not stored any metadata in our current file yet.
f.sections

# Lets check how we can create a new section:
help(f.create_section)

# First we need to create a Section that can contain our annotations
section = f.create_section(name="tag_examples", type_="general_section")

f.sections

# This section can contain further sections as well as properties:
section.sections

section.props

# Lets store additional information about the initial data in our tag example.
sub_sec = section.create_section(name="subject", type_="experiment_A")

# Lets add some properties to this section
help(sub_sec.create_property)

prop = sub_sec.create_property(name="species", values_or_dtype="Mus Musculus")
prop = sub_sec.create_property(name="age", values_or_dtype="4")
prop.unit = "weeks"
prop = sub_sec.create_property(name="subjectID", values_or_dtype="78376446-f096-47b9-8bfe-ce1eb43a48dc")

# Lets check what we have so far:
f.sections

# One section that will describe our tag_examples
f.sections['tag_examples'].sections

# A subsection that contains subject related information
f.sections['tag_examples'].sections['subject'].props

# We can now connect the section describing our tag_example directly to the MultiTag that references both
# the initial as well as the analysed data.

multi_tag = f.blocks['tag_examples'].multi_tags['tag_A']
multi_tag.metadata = f.sections['tag_examples']

# Now when we look at the data via a MultiTag we can directly access all metadata that has been attached to it.
# E.g. get information about the subject the experiment was conducted with
multi_tag.metadata.sections['subject']

# We can also attach the same section to the initial DataArray
init_data = f.blocks['tag_examples'].data_arrays['membrane_voltage_A']
init_data.metadata = f.sections['tag_examples']

# And we can also find it in reverse: we can select a section and find all data, that is connected to it
sec =...


I am working on the file "2019_RDM_course_nix.ipynb".
The first task is:
What we have seen so far:
- we can save different DataArrays that belong to the same experiment in one file in a structured fashion
- we can describe and save different kinds of data to file
- we can add labels and units directly to the data
- we can save multidimensional data
- we can save a bit of space in case of sampled data
- we can better understand the dimensionality of the stored data since we spell out the kind of dimensions which
  makes it easier to interpret it.
Working with multiple data in the same file - tagging points and regions
Saving data together with data of annotation is what you can easily do with Matlab and with additional work in Python as well.
NIX does that and it provides additional features to continue working on this initial data and save the analyzed data in relation to the initial data in a meaningfull way.
- "Tag" regions of interest in a DataArray
- Use the same tag in multiple related DataArrays e.g. in MultiElectrodeArrays
The DataArrays store data, but this is not all that is needed to store scientific data. We may want to highlight points or regions in the data and link it to further information.
This is done using the Tag and the MultiTag, for tagging single or mutliple points or regions, respectively.
The basic idea is that the Tag defines the point (and extent) with which it refers to points (or regions) in the data. A tag can point to several DataArrays at once. These are mere links that are stored in the list of references. The following figure illustrates, how a MultiTag links two DataArrays to create a new construct.
Can you write Python code for it?

# Let us create a new block to illustrate tagged data
block_tag = f.create_block(name="tag_examples", type_="examples")


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
A more advanced example: a carousel.
Can you write Python code for it?

from ipywidgets import Layout, Button, Box, Label

item_layout = Layout(height='100px', min_width='40px')
items = [Button(layout=item_layout, description=str(i), button_style='warning') for i in range(40)]
box_layout = Layout(overflow_x='scroll',
                    border='3px solid black',
                    width='500px',
                    height='',
                    flex_flow='row',
                    display='flex')
carousel = Box(children=items, layout=box_layout)
VBox([Label('Scroll horizontally:'), carousel])


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
Carousel revisted: item layout
The code that generated the carousel is reproduced below. Run the cell, then continue reading.
Can you write Python code for it?


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
Compatibility note
The overflow_x and overflow_y options are deprecated in ipywidgets 7.5. Instead, use the shorthand property overflow='scroll hidden'. The first part specificies overflow in x, the second the overflow in y.
A widget for exploring layout options
Use the dropdowns and sliders in the widget to change the layout of the box containing the colored buttons. Many of the CSS layout options described above are available, and the Python code to generate a Layout object reflecting the settings is in a TextArea in the widget.
A few questions to answer after the demonstration of this (see the detailed flexbox guide for a longer discussion):

What does changing justify_content affect? You may find it easier to answer this if you set wrap to wrap.
What does align_items affect?
How is align_content different than align_items?
Can you write Python code for it?

from layout_preview import layout
layout


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
Flexbox and Grid
The Flexbox CSS specification is great for laying out items in a single direction, horizontally or vertically. As we saw in the previous example, two dimensional layout can be done with flexbox by using a combination of horizontal and vertical components.
The Grid CSS specifation is designed to be used for two dimensional layout. There are properties for specifying the number of items in each row or column, how they should be sized, and how items should be aligned.
For more information about Flexbox and Grid
The are notebooks with more detail about widgets and the Flexbox model and widgets and the Grid model. The code examples from each of those notebooks is included here also.
If you want to learn more about CSS layout after this tutorial, take a look at this excellent set of articles on CSS layout at MDN. The Flexbox and Grid articles each have links to more extensive guides at the end of the article.
The Flexbox layout
The HBox and VBox classes above are special cases of the Box widget.
The Box widget enables the entire CSS flexbox spec as well as the Grid layout spec, enabling rich reactive layouts in the Jupyter notebook. It aims at providing an efficient way to lay out, align and distribute space among items in a container.
Again, the whole flexbox spec is exposed via the layout attribute of the container widget (Box) and the contained items. One may share the same layout attribute among all the contained items.
Acknowledgement
The following flexbox tutorial on the flexbox layout follows the lines of the article A Complete Guide to Flexbox by Chris Coyier, and uses text and various images from the article with permission.
Basics and terminology
The flexbox layout spectrum is excellent for laying out items in a single direction, either horizontally or vertically.
Since flexbox is a whole module and not a single property, it involves a lot of things including its whole set of properties. Some of them are meant to be set on the container (parent element, known as "flex container") whereas...
Can you write Python code for it?

from ipywidgets import Layout, Button, Box

items_layout = Layout(width='auto')     # override the default width of the button to 'auto' to let the button grow

box_layout = Layout(display='flex',
                    flex_flow='column',
                    align_items='stretch',
                    border='solid',
                    width='50%')

words = ['correct', 'horse', 'battery', 'staple']
items = [Button(description=word, layout=items_layout, button_style='danger') for word in words]
box = Box(children=items, layout=box_layout)
box


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
Is simple layout really simple?
The cell below adds a min_width and max_width to the button layout. The effect may be surprising; in CSS if max/min width are present they override the width.
Can you write Python code for it?


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
The first example defines a 3x3 grid and places 9 buttons into the grid.
Can you write Python code for it?

GridBox(children=[Button(description=str(i), layout=Layout(width='auto', height='auto'),
                         style=ButtonStyle(button_color='darkseagreen')) for i in range(9)
                 ],
        layout=Layout(
            width='50%',
            grid_template_columns='100px 50px 100px',
            grid_template_rows='80px auto 80px',
            grid_gap='5px 10px')
       )


I am working on the file "2020-01-09-PyData-Heidelberg/examples/06.01-Widget_Layout.ipynb".
The first task is:
To do:

Change the min_width for one of the items, say the first one. Does it affect only the first one, or all of them? Why?
Change the height of only the first button. Hint: It needs its own Layout.
Can you write Python code for it?

items[0].layout.min_width = 'FILL IN WITH A WIDTH'


I am working on the file "2020-01-09-PyData-Heidelberg/examples/Lorenz.ipynb".
The first task is:
After interacting with the system, we can take the result and perform further computations. In this case, we compute the average positions in \(x\), \(y\) and \(z\).
Can you write Python code for it?

xyz_avg = x_t.mean(axis=1)

xyz_avg.shape


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
2x2 Grid
You can easily create a layout with 4 widgets arranged in a 2x2 matrix using the TwoByTwoLayout widget:
Can you write Python code for it?

from ipywidgets import TwoByTwoLayout

layout = dict(height='300px')

TwoByTwoLayout(top_left=top_left_button,
               top_right=top_right_button,
               bottom_left=bottom_left_button,
               bottom_right=bottom_right_button,
               layout=layout)


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
For other alignment options it's possible to use common names (top and bottom) or their CSS equivalents (flex-start and flex-end):
Can you write Python code for it?

TwoByTwoLayout(top_left=top_left_button, top_right=top_right_button,
               bottom_right=bottom_right_button,
               justify_items='center',
               width="50%",
               align_items='top')


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
Grid layout
GridspecLayout is a N-by-M grid layout allowing for flexible layout definitions using an API similar to matplotlib's GridSpec.
You can use GridspecLayout to define a simple regularly-spaced grid. For example, to create a 4x3 layout:
Can you write Python code for it?

from ipywidgets import GridspecLayout

grid = GridspecLayout(4, 3, layout=layout)

for i in range(4):
    for j in range(3):
        grid[i, j] = create_expanded_button('Button {} - {}'.format(i, j), 'warning')
grid


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
However with the automatic merging feature, it's possible to achieve many other layouts:
Can you write Python code for it?

AppLayout(header=None,
          left_sidebar=None,
          center=center_button,
          right_sidebar=None,
          footer=None,
          layout=layout)

AppLayout(header=header_button,
          left_sidebar=left_button,
          center=center_button,
          right_sidebar=right_button,
          footer=None,
          layout=layout)

AppLayout(header=None,
          left_sidebar=left_button,
          center=center_button,
          right_sidebar=right_button,
          footer=None,
          layout=layout)

AppLayout(header=header_button,
          left_sidebar=left_button,
          center=center_button,
          right_sidebar=None,
          footer=footer_button,
          layout=layout)

AppLayout(header=header_button,
          left_sidebar=None,
          center=center_button,
          right_sidebar=right_button,
          footer=footer_button,
          layout=layout)

AppLayout(header=header_button,
          left_sidebar=None,
          center=center_button,
          right_sidebar=None,
          footer=footer_button,
          layout=layout)

AppLayout(header=header_button,
          left_sidebar=left_button,
          center=None,
          right_sidebar=right_button,
          footer=footer_button,
          layout=layout)


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
Note: Slices are supported in this context.
Can you write Python code for it?

grid[:3, 1:] = create_expanded_button('I am new too!!!!!', 'warning')


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
Style attributes
You can specify extra style properties to modify the layout. For example, you can change the size of the whole layout using the height and width arguments.
Can you write Python code for it?

AppLayout(header=None,
          left_sidebar=left_button,
          center=center_button,
          right_sidebar=right_button,
          footer=None,
          height="200px", width="50%")


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
The gap between the panes can be increase or decreased with grid_gap argument:
Can you write Python code for it?

AppLayout(header=None,
          left_sidebar=left_button,
          center=center_button,
          right_sidebar=right_button,
          footer=None,
          height="200px", width="50%",
          grid_gap="10px")


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
To make a widget span several columns and/or rows, you can use slice notation:
Can you write Python code for it?

grid = GridspecLayout(4, 3, layout=layout)
grid[:3, 1:] = create_expanded_button('One', 'success')
grid[:, 0] = create_expanded_button('Two', 'info')
grid[3, 1] = create_expanded_button('Three', 'warning')
grid[3, 2] = create_expanded_button('Four', 'danger')
grid


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
You can add a missing widget even after the layout initialization:
Can you write Python code for it?

layout_2x2 = TwoByTwoLayout(top_left=top_left_button,
                            bottom_left=bottom_left_button,
                            bottom_right=bottom_right_button,
                            layout=layout)
layout_2x2

layout_2x2.top_right = top_right_button


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
You can also use the linking feature of widgets to update some property of a widget based on another widget:
Can you write Python code for it?

app = TwoByTwoLayout(top_left=top_left_text, top_right=top_right_text,
                     bottom_left=bottom_left_slider, bottom_right=bottom_right_slider)

link_left = jslink((app.top_left, 'value'), (app.bottom_left, 'value'))
link_right = jslink((app.top_right, 'value'), (app.bottom_right, 'value'))
app.bottom_right.value = 30
app.top_left.value = 25
app


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridlayout.ipynb".
The first task is:
You can pass merge=False in the argument of the TwoByTwoLayout constructor if you don't want this behavior
Can you write Python code for it?

TwoByTwoLayout(top_left=top_left_button,
               bottom_left=bottom_left_button,
               bottom_right=bottom_right_button,
               merge=False,
               layout=layout)


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridstack.ipynb".
The first task is:
Load Data Top
Can you write Python code for it?


I am working on the file "2020-01-09-PyData-Heidelberg/examples/gridstack.ipynb".
The first task is:
We also need a function on_pick_scotch that will display a table of the top 5 similar scotches that Radar View watches, based on a given selected Scotch.
Can you write Python code for it?

def on_pick_scotch(Scotch):
    name = Scotch
    # Get top 6 similar whiskeys, and remove this one
    top_df = get_similar(name, 6).iloc[1:]
    # Get bottom 5 similar whiskeys
    df = top_df

    # Make table index a set of links that the radar widget will watch
    df.index = ['''{}'''.format(name, i, i) for i in df.index]

    tmpl = f'''
If you like {name} you might want to try these five brands. Click one to see how its taste profile compares.
'''
    prompt_w.value = tmpl
    table.value = df.to_html(escape=False)
    radar_w.factors_keys = [name]
    plot = radar_w.factors_keys_changed()

prompt_w = widgets.HTML(value='Aberfeldy')
display(prompt_w)

table = widgets.HTML(
    value="Hello World"
)
display(table)

radar_w = RadarWidget(df=features_df)

picker_w = widgets.interact(on_pick_scotch, Scotch=list(sim_df.index))

radar_w.factors_keys


I am working on the file "2020-01-09-PyData-Heidelberg/examples/reveal.ipynb".
The first task is:
loading the iris dataset
Can you write Python code for it?

iris = px.data.iris()

iris.head()

fig = go.FigureWidget()
keys = list(iris.keys()[:4])

@widgets.interact(x=keys, y=keys[::-1])
def update_px(x, y):
    p = px.scatter(iris, x, y, color='species', width=800, height=600)
    for i in range(len(p.data)):
        fig.data = []
    fig.update(data = [d.to_plotly_json() for d in p.data])
    fig.plotly_relayout(p.layout.to_plotly_json())

fig


I am working on the file "2020-01-09-PyData-Heidelberg/presentation.ipynb".
The first task is:
Rich ecosystem of third-party widgets
- bqplot: Interactive 2-D plotting
- ipyleaflet: Interactive maps
- ipycanvas: Interactive Canvas
- ipyvolume: Interactive 3-D plotting
- ipywebrtc, ipysheet, ipytree ...

bqplot
Can you write Python code for it?

import ipywidgets as widgets
import numpy as np

from bqplot import pyplot as plt

n = 2000
x = np.linspace(0.0, 10.0, n)
np.random.seed(0)
y = np.cumsum(np.random.randn(n)*10).astype(int)

fig_hist = plt.figure(title='Histogram')
hist = plt.hist(y, bins=25)
fig_hist

slider = widgets.IntSlider(min=1, max=100, v_model=30)
widgets.link((hist, 'bins'), (slider, 'value'));
slider


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
A representação padrão de lista em string é uma sequência das strings de seu conteúdo, separado por ', ' e cercados por colchetes. O método join, de strings, existe para usar uma string como separador de uma lista, conforme o exemplo abaixo
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Atribuição ternário
Python também conta com atribuição ternária
Can you write Python code for it?

x = 5
par_ou_impar = "par" if x % 2 == 0 else "impar"
print(par_ou_impar)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Combinando transformação e filtragem: Compressão de Listas
Uma maneira prática para combinar as operações de transformação e filtragem é utilizar uma compressão de lista. A anatomia da compressão de lista é a seguinte:
[f(x) for x in lista if p(x)]
Can you write Python code for it?

quadrados = [x * x for x in range(5)]
print(quadrados)

quadrados_dos_pares = [x * x for x in range(5) if x%2 == 0]
print(quadrados_dos_pares)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Data
Em Python, podemos manipular informações de data, hora e tempo utilizando o módulo datetime
Can you write Python code for it?

from datetime import datetime
import time

inicio = datetime.now() # obtém o timestamp do momento atual
print(inicio)
for i in range(2):
    time.sleep(1) # aplica um delay, em segundos
fim = datetime.now()
print(fim)
print("Tempo de execução: {}".format(fim - inicio))


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Dicionários
Dicionários são pares de chave : valor, utilizados para representar um dado cujos valores tem uma semântica atribuída a ela.
Can you write Python code for it?

dicionario = { 'nome': 'Fulano', 'sobrenome': 'Da Silva' }
print(dicionario['nome'])
print(dicionario['sobrenome'])


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Em Python, as listas são mutáveis, ou seja, ela pode ser modificada.
Can you write Python code for it?

lista.append(7) # adiciona elemento na lista
lista.append(6)
lista.append(7)
print(lista)

lista.remove(7) # remove o primeiro elemento cujo valor é 7
print(lista)

lista.extend([8, 9, -10]) #extende a lista com outra lista
lista


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Em Python, podemos acessar, separadamente, suas chaves e seus valores
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Embora a lista seja mutável, as operações de acesso aos elementos (com o operador [ ]) retornam novos elementos. Logo, podemos fazer uma cópia de uma lista da seguinte maneira
Can you write Python code for it?

lista3 = lista[:]  # lista3 é uma nova lista com o mesmo conteúdo de lista
lista3[-1] = -10
print(lista)
print(lista3)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Entretanto, o for funciona iterando sobre uma coleção/sequência de valores. Existem diversas maneiras de fazer isso:
Can you write Python code for it?

# definindo um intervalo

for i in range(10): # de 0 a 9, pulando de 1 em 1
    print(i, end= " ")

print()

for i in range(5,10): # de 5 a 9, pulando de 1 em 1
    print(i, end= " ")

print()

for i in range(0,10,2): # de 0 a 9, pulando de 2 em 2
    print(i, end= " ")

print()

# utilizando uma sequência pre-definida
a = [1, 3, 4, 5, 7]
for i in a:
    print(i)

# a função enumerate retorna uma lista de pares (i,x), onde:
# i é o índice (0, 1, 2, ...)
# x é o elemento da sequência original
a = [1, 3, 4, 5, 7]
for indice, valor in enumerate(a):
    print(indice, valor)

# podemos também percorrer uma coleção mas não utilizar o valor daquele passo
for _ in range(5):
    print('oi')

# existem também dois operadores dentro de um for:
# continue - encerra o passo atual e passa para o próximo
# break - encerra o laço
for x in range(10):
    if x == 3:
        continue
    if x == 5:
        break
    print(x)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Existe também o operador de potência, permitindo elevar um número ao outro.
PS: como as raízes também são potências (porém, invertidas), podemos computá-las com o mesmo operador
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Existem algumas maneiras de interpolar strings, ou seja, injetar valores dentro dela
Can you write Python code for it?

# %d e %f indicam, respectivamente, que um número inteiro ou ponto flutuante será inserido
print("%d/%f/%d" % (2, 4.5, 6))
print("%s/%s/%s" % ('a', 2, False)) # %s indica que algo será inserido como string


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Existem diversas maneiras de acessar os elementos de uma lista:
Can you write Python code for it?

lista[2] #elemento da lista

lista[-1] # último elemento

lista[1:3] #sublista

lista[:3] # 3 primeiros elementos

lista[-2:] # 2 últimos elementos

lista[1:-1] # do segundo ao penúltimo elemento


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Existem também as funções anônimas (chamadas, em Python, de lambda). Uma função anônima é uma função que contém apenas a assinatura e o corpo, sem um nome definido. Em Python, elas recebem o nome de lambda pois elas são funções de um comando só (diferente de linguagens como Javascript e Scala, que permitem funções anônimas de múltiplas linhas).
Can you write Python code for it?

chamar_funcao(lambda : print("Lambda function"))


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Fato interessante: strings em python podem ser manipuladas com operadores de soma (concatenação) ou multiplicação (repetição)
Can you write Python code for it?

print("Big" + " " + "Data")
print('Repete ' * 5)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Funções
Para definir uma função em python, basta utilizarmos a seguinte estrutua:
def nome_da_função (argumento1, argumento2, ...):
    corpo da função
Can you write Python code for it?

def soma(a, b):
    return a + b

print(soma(3, 5))
print(soma('casa ', 'organizada'))


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Laços / Loops
Em Python, a estrutura de repetição while funciona de maneira análoga a diversas outras linguagens:
Can you write Python code for it?

x = 0
while x < 5:
    print(x)
    x += 1


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Listas
Quando desejamos trabalhar com uma sequência de dados, podemos utilizar uma lista
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Novamente temos um iterator como retorno. Podemos utilizar as mesmas abordagens do iteratoor apresentado no map
Can you write Python code for it?

#descomente uma alternativa para checar seu funcionamento

even_a = filter(lambda x: x%2 == 0, a)

# for x in even_a:
#     print(x, end=" ")

# str_even_a = map(lambda x: str(x), even_a)
# print(" ".join(str_even_a))

# list(even_a)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Números
Em python, existem dois tipos básicos de números: Inteiros (int) e Ponto Flutuante (float)
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
O módulo random contém também um conjunto de funções para trabalharmos com listas. Podemos embaralhá-las, escolher um elemento aleatório ou obter uma amostra de tamanho n
Can you write Python code for it?

import random

a = ['a', 'casa', 'está', 'muito', 'bem', 'organizada']

random.shuffle(a) # mutável
print(a)
random.shuffle(a)
print(a)

random.choice(a) # com repetição

numeros = range(1, 100)
random.sample(numeros, 5) # sem repetição


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
O operador len pode ser utilizado para medir tamanhos de objetos com múltiplas informações (strings, lista, etc.)
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
OBS: se um elemento da lista não for uma string, o método join não funciona
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Operações funcionais sobre listas
Existem algumas funções primitivas para operar sobre os elementos de uma lista. Duas delas são transformação (map) e filtragem (filter).
Transformação
O objetivo desta função é, como o próprio nome sugere, transformar os elementos de uma lista, gerando uma nova lista. Em termos mais matemáticos: dada uma função f(x) definida para todos os elementos de uma lista L [a1, a2, ..., an], será gerada uma nova lista L2 [b1, b2, ..., bn] tal que bi = f(ai).
Para realizar uma trnasformação, utilizamos a função map
Can you write Python code for it?

a = [1, 2, 3, 4, 5]

doubled_a = map(lambda x: 2*x, a)
doubled_a


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Outra maneira de interpolar strings é usando o format
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Outra operação interessante de limpeza de strings é o strip, o qual remove os espaços em branco no começo e no final da string
Can you write Python code for it?

print(' Hello '.strip())  # remove no começo e no final
print(' Hello '.lstrip()) # remove no começo
print(' Hello '.rstrip()) # remove no final


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Podemos facilmente substituir a ocorrência de um caractere/substring utilizando o método replace
Can you write Python code for it?

str1 = 'Hoje_vai_chover_novamente.'.replace('_', ' ')
print(str1)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Podemos também consultar a pertinência de um elemento em uma lista utilizando o operador in
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Por padrão, as variáveis em Python (quando apontando para tipos de dados mais complexos) são ponteiros, ou seja, elas são uma referência a um dado alocado na memória. Logo, quando salvamos o valor de uma variável A em uma variável B, estamos apenas dizendo que B aponta para a mesma informação que A
Can you write Python code for it?

lista2 = lista  # lista2 referencia lista
lista2[-1] = 10
print(lista)
print(lista2)


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Python também conta um módulo de funções para operações randômicas
Can you write Python code for it?

import random
print(random.random())        # normalizado entre 0 e 1
print(random.randint(1,10))   # intervalo fechado


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Python, assim como muitas linguagens, conta com um módulo que implementa diversas funções matemáticas (raíz, logaritmo, topo, piso, etc.
Can you write Python code for it?

import math
print(math.sqrt(2))
print(math.log(2))
print(math.ceil(2 ** 0.5))
print(math.floor(2 ** 0.5))


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Quando temos strings que contém múltiplas informações e desejamos separá-la, podemos usar a função split, informando o caractere/substring a ser utilizado para dividir a string
Can you write Python code for it?


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Strings
Em Python, trabalhar com strings é bem simples, uma vez que já existem diversas funções implementadas.
Para criar uma string em Python, basta colocar o conteúdo entre aspas, simples ou duplas (o tipo de aspas que começa a string é o mesmo tipo que irá finalizá-la)
Can you write Python code for it?

single_quoted_string = 'Data Science'
print(single_quoted_string, type(single_quoted_string))

print(len(single_quoted_string))    # tamanho da string
print(single_quoted_string.upper()) # colocar caracteres em maiusculo
print(single_quoted_string.lower()) # colocar caracteres em minusculo


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Também podemos criar string mais longas, com múltiplas linhas, utilizando um total de 6 aspas (3 para abrir e 3 para fechar)
Can you write Python code for it?

multi_line_string = """linha 1
linha 2
linha 3"""
print(multi_line_string)
print(repr(multi_line_string))


I am working on the file "2020/01-python-jupyter-notebook/Python-101.ipynb".
The first task is:
Tuplas
Tuplas são grupos ordenados de valores. Esse par é imutável e normalmente é utilizado para representar informações estruturadas (linha de um csv, tupla de um banco de dados, etc)
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Matplotlib.ipynb".
The first task is:
Integração com o Pandas
O Pandas já possui um conjunto de operações que permitem a geração de gráficos a partir de Series e DataFrames!
Can you write Python code for it?

import pandas as pd
df = pd.read_csv('titanic.csv')
df


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Matplotlib.ipynb".
The first task is:
Os resultados de operações mais complexas sobre um DataFrame também podem ser plotados em gráficos!
Can you write Python code for it?

df['Survived'].apply(lambda s: "Yes" if s == 1 else "No").value_counts().plot(kind='bar')


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Matplotlib.ipynb".
The first task is:
Para a geração de simples gráficos de linhas, basta utilizarmos o método plot
Can you write Python code for it?

y = [1, 7, 3, 5, 12]
x = [1, 2, 3, 4, 5]
plt.plot(x, y, marker='o');

plt.plot(x, y, marker='x', c='r') # gráfico vermelho ('r') onde os pontos são 'x'
plt.grid() # adiciona as guias no gráfico


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Matplotlib.ipynb".
The first task is:
Para gerar gráficos de dispersão (scatter plot), podemos utilizar o método scatter
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Numpy.ipynb".
The first task is:
Existem também funções para gerar arrays pré-inicializados
Can you write Python code for it?

print(np.zeros((3, 5)), end="\n\n") # array de zeros com dimensões [3,5]
print(np.ones((2,3,4)), end="\n\n------------\n\n") # array de uns com dimensões [2,3,4]
print(np.full((2, 2), 10), end="\n\n") # array de 10 com dimensões [2,2]
print(np.arange(10, 30, 5), end="\n\n") # valores de 10 a 30 com passo 5
print(np.random.rand(2, 3), end="\n\n") # array cde dimensao [2,3] com valores aleatórios


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Numpy.ipynb".
The first task is:
Existem também operações utilitárias pré-definidas em um array
Can you write Python code for it?

print(b,end="\n\n")
print('Axis 1: %s' % b[0], end="\n\n") # retorna um vetor
print(np.average(b), end="\n\n") # tira a média dos elementos
print(np.average(b, axis=1), end="\n\n") # tira a média dos elementos dos vetores no eixo 1
print(b.sum(), end="\n\n") # retorna as somas dos valores
print(b.sum(axis=1), end="\n\n") # retorna as somas dos valores no eixo 1
print(b.min(), end="\n\n") # retorna o menor valor
print(b.max(), end="\n\n") # retorna o maior valor


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Numpy.ipynb".
The first task is:
O Numpy conta também com funções para salvar/ler arrays de arquivos
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Numpy.ipynb".
The first task is:
O Numpy traz consigo diversas operações matemáticas implementadas, as quais podem ser aplicadas sobre um valor ou um array de valores.
OBS: podemos ver a aplicações dessas funções como uma operação de transformação (map)
Can you write Python code for it?

print(10*np.sin(1)) # seno trigonométrico de 1
print(10*np.sin(a)) # seno trigonométrico de cada elemento de a


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Numpy.ipynb".
The first task is:
Podemos selecionar intervalos do array, permitindo recuperar apenas uma porção dele
Can you write Python code for it?

d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
d

d[:, 0] # todas as linhas (:) da primeira coluna (0)

d[:, 1] # todas as linhas (:) da segunda coluna (1)

d[:, 0:2] # todas as linhas (:) das colunas de 0 à 2

d[:, 2] # todas as linhas (:) da terceira coluna (2)


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
Além disso, o índice pode ser utilizado para dar uma semântia ao dado de uma série, permitindo também que o mesmo seja acessado pelo índice atribuído
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
As Series também possuem um método de transformação, conforme apresentado na aula anterior. Esse método chama-se apply: ele recebe uma função que será aplicada sobre todos os elementos da Serie, retornando então uma Serie com os resultados
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
DataFrame
Um DataFrame é uma tabela onde cada coluna é uma Serie. Assim como as Series, o DataFrame possui um índice, porém, o índice refere-se a uma linha inteira, ou seja, ao elemento naquela posição em todas suas colunas
Can you write Python code for it?

matriz = np.array([[1, 2, 3], [4, 5, 6]])
nomes_linhas = ['L1', 'L2']
nomes_cols = ['C1', 'C2', 'C3']

df = pd.DataFrame(matriz, index=nomes_linhas, columns=nomes_cols)
df


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
Enquanto em uma Serie utilizamos os colchetes ([]) para acessar um elemento em um certo índice, no DataFrame o operador refere-se à uma Serie, permitindo acessá-la, sobrescrevê-la ou adicionar uma nova
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
O DataFrame permite também que sejam relizadas contagens sobre os valores presentes nas séries, permitindo assim analizarmos a ocorrência de certos dados categóricos
Can you write Python code for it?

df['Sex'].value_counts()

df['Survived'].apply(lambda s: "Yes" if s == 1 else "No").value_counts()


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
O operador colchetes ([]) do pandas também pode ser usado como uma filtragem, ou seja, dada uma condição (ou predicado), ele retorna apenas as linhas do DataFrame que satisfaçam o predicado.
OBS: escrever um predicado nesse operador nem sempre é tão simples quanto um predicado Python comum
Can you write Python code for it?


I am working on the file "2020/02-python-bibliotecas-manipulacao-dados/Pandas.ipynb".
The first task is:
Para fins de exportação, um DataFrame pode ser representado em diversos formatos
Can you write Python code for it?

print(df.to_latex()) # latex

print(df.to_csv(index=False)) #csv

print(df.to_json()) # JSON

print(df.to_html()) #HTML


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
1.1 Clusterização Hierárquica
Verifique abaixo o resultado da clusterização hierárquica variando a quantidade de clusters de forma iterativa. Teste também outras funções de ligação (linkage).
Fique livre parar alterar ou criar novos dados no dataset de exemplo abaixo.
Can you write Python code for it?

# Dataset
tmp_data = np.array([[1,1],[2,2],[2,3],[4,4],
                     [4,2],[6,6],[10,9],[7,7],
                     [8,8],[7,9],[10,10],[14,2]])

plt.scatter(tmp_data[:,0], tmp_data[:,1], s=150)
plt.show()

# Teste diferentes funções de ligações
# {“ward”, “complete”, “average”}
linkage = 'ward'

# Clusterização hierárquica
ag_model = AgglomerativeClustering(linkage=linkage)

# Número de colunas do plot
plot_col = 3

n_rows, n_col = tmp_data.shape
fig, ax = plt.subplots(nrows=math.ceil(len(tmp_data)/plot_col),
                       ncols=plot_col,figsize=(20,20))

count = 0
for n_clusters in range(len(tmp_data),0,-1):
    index = abs(n_clusters - len(tmp_data))

    # Atribui o número de clusters
    ag_model.n_clusters = n_clusters
    # Ajusta o modelo
    ag_model = ag_model.fit(tmp_data)

    if plot_col == 1:
        ax[count].scatter(tmp_data[:,0], tmp_data[:,1], c=ag_model.labels_, s=150)
        ax[count].set_title("Qt. Clusters: " + str(n_clusters))
    else:
        ax[count, (index)%plot_col].scatter(tmp_data[:,0], tmp_data[:,1], c=ag_model.labels_, s=150)
        ax[count, (index)%plot_col].set_title("Qt. Clusters: " + str(n_clusters))

    if (index+1) % plot_col == 0:
        count += 1

plt.show()


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
1.2 Dendrograma
Um dendrograma é um tipo de diagrama de árvore que mostra o relacionamentos entre conjuntos de dados semelhantes, ou agrupamento hierárquico. Eles são freqüentemente usados em biologia para mostrar o agrupamento entre genes ou amostras, mas podem representar qualquer tipo de dados agrupados.

https://www.statisticshowto.datasciencecentral.com/hierarchical-clustering/

Utilize o código abaixo para criar os Dendrogramas dos dados utilizados nas células acima. Fique livre para retornar e testar outras distribuições dos dados e funções de ligamento (linkage).
Can you write Python code for it?

# Usa o método de linkage especificado
# para construir o dendrograma
if (linkage == 'average'):
    Z = hierarchy.average(tmp_data)
elif (linkage == 'complete'):
    Z = hierarchy.complete(tmp_data)
elif (linkage == 'ward'):
    Z = hierarchy.ward(tmp_data)

plt.figure()
plt.title("Dendrograma - linkage: " + str(linkage))
dn = hierarchy.dendrogram(Z)


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Mostre e análise o dendrograma do exemplo acima.
Can you write Python code for it?


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Os dados parecem estar agrupados corretamente? Tente fazer o mesmo procedimento com algoritmo de clusterização hieráquica

https://scikit-learn.org/stable/modules/generated/sklearn.cluster.AgglomerativeClustering.html
Can you write Python code for it?


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Os dados parecem estar agrupados corretamente? Tente fazer o mesmo procedimento com algoritmo de clusterização hieráquica, altera os parâmetros se for necessário.
Can you write Python code for it?


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Qual a sua conclusão?
"Escreva aqui"
1.4 Exemplo 2 - Noisy Circles
Can you write Python code for it?

# Gerar o conjunto de dados
n_samples = 1500
noisy_circles = datasets.make_circles(n_samples=n_samples, factor=.5,
                                      noise=.05)
dataset_circles = noisy_circles[0]

# Mostrar os dados
plt.scatter(dataset_circles[:,0], dataset_circles[:,1])
plt.show()


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Utilize o algoritmo do K-means para separa os conjuntos.
Can you write Python code for it?


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_Clustering_Assignment.ipynb".
The first task is:
Utilize o algoritmo do K-means para separa os conjuntos.
Você pode importar o K-means criado por você! Para importar uma função de um notebook para outro instale a biblioteca nbimporter:
 - pip install nbimporter
Can you write Python code for it?

import nbimporter
from Notebook_KMeans import KMeans

### CODE HERE ###


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_WhyNumpy.ipynb".
The first task is:
Executar a função com for loop
Can you write Python code for it?

start = time.time()

# Executar 200x o código analisado
for aux in range(200):

    #----------------- code analysed ------------
    all_sum = 0
    for i in range(len(x)):
        if(x[i] in k):
            all_sum += abs(x[i]*x[i] - 2*x[i])
    #--------------------------------------------

end = time.time()
time_for = end - start
print('Resultado da equação: ', all_sum)
print('Tempo de execução com \"for\": {0:.2f}s'.format(time_for))


I am working on the file "2020/04-unsupervised-learning-clustering/Notebook_WhyNumpy.ipynb".
The first task is:
Executar a função com numpy
Can you write Python code for it?

start = time.time()

# Executar 200x o código analisado
for aux in range(200):

    #----------------- code analysed ------------
    np_result = np_x[np.isin(np_x,k)]
    np_result = abs(np.square(np_result) - 2*np_result)
    all_sum_np = np.sum(np_result)
    #--------------------------------------------

end = time.time()
time_np = end - start
print('Resultado da equação: ', all_sum_np)
print('Tempo de execução com \"numpy\": {0:.2f}s'.format(time_np))

print('Para este exemplo a execução com numpy foi {0:.0f}x mais rápida que utilizando for'.format(time_for/time_np))


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Agora você deve ser capaz de imprimir as respostas para as seguintes perguntas:

Qual foi a menor distância percorrida em metros?
Qual foi a distância média percorrida em metros?
Qual foi a distância máxima percorrida em metros?
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Em seguida:
- Crie um GeoDataFrame vazio chamado movements
- Para cada usuário:
     - ordene as linhas por timestamp;
     - crie um objeto LineString baseado nos posts do usuário;
     - adicione a geometria e o userid no GeoDataFrame que você criou na última etapa;
- Defina o CRS dos movements GeoDataFrame como EPSG: 32735 (código epsg: 32735)
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Por último:
- Crie um mapa simples dos pontos (um deles é suficiente):
     - Crie o mapa usando a função .plot () no Python. Você pode precisar usar o método matplotlib.pyplot.show() para exibir a imagem dentro deste bloco de notas.
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Por último:
- salve o polígono em um arquivo Shapefile chamado 'polygon.shp'.
- mostrar o polygono usando a função 'plot'
Can you write Python code for it?

## CODE HERE ##

# Plot a map of the polygon

# REPLACE THE ERROR BELOW WITH YOUR OWN CODE
## CODE HERE ##


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Problem 3: o quanto de distância as pessoas viajaram?
Neste problema, o objetivo é calcular a distância em metros que os indivíduos percorreram de acordo com os
posts nas redes sociais (distâncias euclidiana entre os pontos).
Usaremos o shapefile "Kruger_posts.shp" gerado no Problema 2 como entrada.
Você pode seguir os seguintes passos:
  - leia o arquivo shapefile como um geodataframe
  - Reprojetar os dados da projeção WGS84 para a projeção EPSG: 32735 que significa Zona UTM 35S (zona UTM para a África do Sul) para transformar os dados em sistema métrico.
  - Agrupar os dados pelo ID do usuário.
Can you write Python code for it?

## CODE HERE ##

# Check that the crs is correct (should be epsg:32735)
print(data.crs)

## CODE HERE ##


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Problema 2: operação de pontos com mapa
O problema 2 continua o processo que começamos no Exercício 1, ou seja, criar objetos de ponto geométrico e colocá-los em um mapa.
Neste problema, nosso objetivo é plotar um mapa baseado em um conjunto de coordenadas x e y que são armazenadas em um arquivo csv.
As coordenadas estão em graus decimais WGS84, e os dados são armazenados no arquivo separado por vírgulas some_posts.csv na pasta data. As primeiras linhas dos dados são assim:
lat,lon,timestamp,userid
-24.980792492,31.484633302,2015-07-07 03:02,66487960
-25.499224667,31.508905612,2015-07-07 03:18,65281761
-24.342578456,30.930866066,2015-03-07 03:38,90916112
-24.85461393,31.519718439,2015-10-07 05:04,37959089
Os dados têm 81379 linhas e consistem em locais e horários de postagens nas redes sociais dentro do parque nacional Kruger na África do Sul:
| Column | Description |
|--------|-------------|
| lat | y-coordinate of the post |
| lon | x-coordinate of the post |
| timestamp | Time when the post was uploaded |
| userid | userid |
Nota: embora os dados sejam baseados em dados reais de mídias sociais, eles estão anônimos. Os dados de userids e timestamps foram randomizados, ou seja, eles não coincidem com os dados reais, também a precisão espacial dos dados foi reduzida.
Primeiro:

Importe os módulos necessários
Leia os dados de data / some_posts.csv em um dataframe do Pandas chamadodata
Crie uma coluna vazia chamada geometry onde você irá armazenar objetos Point bem formados
Inserir objetos Point na coluna geometry baseada nas colunas de coordenadas
    - Você pode querer iterar sobre as linhas do dataframe e usar o indexador .loc para atualizar o valor da coluna correta em cada linha. Veja mais dicas no notebook sobre GeoPandas
    - Nota: se esta etapa estiver demorando muito para ser executada, você pode primeiro testar seu código com um subconjunto dos dados originais, por exemplo. pegando as primeiras 50 linhas de dados data [: 50]. Depois que o código funcionar corretamente, você poderá executar o processo...
Can you write Python code for it?

## CODE HERE ##

# Check the dataframe head
print(data.head())


I am working on the file "2020/05-geographic-information-system/Notebook_Exercise-2.ipynb".
The first task is:
Próximo:
- Converta esse DataFrame em um GeoDataFrame;

Atualize o CRS para o sistema de coordenadas como WGS84 (ou seja, código epsg: 4326);

Salve os dados em um Shapefile chamado Kruger_posts.shp.
Can you write Python code for it?

## CODE HERE ##

# Check the geodataframe head
print(geo.head())


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
2. Objeto LineString (Linha)
Criar um objeto LineString é relativamente similar de como o objeto Point foi criado.
Agora em vez de usar uma única tupla de coordenadas, nós podemos contruir uma linha usando uma lista de Point ou um vetor de tuplas com as respectivas coordenadas:
Can you write Python code for it?

# Create a LineString from our Point objects
line = LineString([point1, point2, point3])

# It is also possible to produce the same outcome using coordinate tuples
line2 = LineString([(2.2, 4.2), (7.2, -25.1), (9.26, -2.456)])


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Como mostrado acima podemos ver que o objeto MultiPolygon possui os mesmos atributos que o objeto Polygon, mas agora métodos como cálculo da área retorna a soma das áreas de todos os polígonos presentes no objeto.
Também existem algumas funções extras disponíveis apenas para estas coleções, como is_valid que retorna se o polígono ou linhas possuem interseção um com o outro.
Can you write Python code for it?

valid = multi_poly.is_valid
print("Is polygon valid?: ", valid)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Como pode ser observado o tipo de dado da variável point_coords é um Shapely CoordinateSequence.
Vamos ver como recuperar as coordenadas deste objeto:
Can you write Python code for it?

# Get x and y coordinates
xy = point_coords.xy

# Get only x coordinates of Point1
x = point1.x

# Whatabout y coordinate?
y = point1.y

# Print out
print("xy:", xy, "\n")
print("x:", x, "\n")
print("y:", y)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Como podemos observar, as coordenadas novamente são armazenadas em arrays numpy, em que o primeiro array inclui todas as coordenadas x e o segundo array todas as coordendas y.
Podemos extrair somente as coordendas x e y das seguintes formas.
Can you write Python code for it?

# Extract x coordinates
line_x = lxy[0]

# Extract y coordinates straight from the LineObject by referring to a array at index 1
line_y = line.xy[1]

print('line_x:\n', line_x, '\n')

print('line_y:\n', line_y)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Como podemos observar, o Polygon agora possui duas tuplas diferentes de coordendas. A primeira representa o exterior e a segunda representa o buraco presente no interior do polígono.
3.1 Objeto Polygon - Atributos e funções
Podemos novamente acessar diferentes atributos diretamente do objeto Polygon, que podem ser bastante úteis para muitas análises, como: obter área, centróide, bounding box, o exterior e o perímetro (tamanho exterior).
Aqui, podemos ver algunas atributos disponíveis e como acessá-los:
Can you write Python code for it?

# Get the centroid of the Polygon
poly_centroid = poly.centroid

# Get the area of the Polygon
poly_area = poly.area

# Get the bounds of the Polygon (i.e. bounding box)
poly_bbox = poly.bounds

# Get the exterior of the Polygon
poly_ext = poly.exterior

# Get the length of the exterior
poly_ext_length = poly_ext.length

# Print the outputs
print("Poly centroid: ", poly_centroid)
print("Poly Area: ", poly_area)
print("Poly Bounding Box: ", poly_bbox)
print("Poly Exterior: ", poly_ext)
print("Poly Exterior Length: ", poly_ext_length)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Outros atributos internos da coleção de geometria:
Can you write Python code for it?

# How many lines do we have inside our MultiLineString?
lines_count = len(multi_line)

# Print output:
print("Number of lines in MultiLineString:", lines_count)

# Let's calculate the area of our MultiPolygon
multi_poly_area = multi_poly.area


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Perceba que a representação do Polygon possui dois parentesis ao redor das coordendas (ex.: POLYGON ((<valores>)) ). Isso acontece porque o objeto Polygon pode ter aberturas (buracos) dentro dele.
Como é mostrado na documentação do Polygon (utilizando a função help do python) um polígono pode ser construído usando as coordendas exteriores e coordendas interiores (opcionais), em que as coordendas interiores criam um buraco dentro do polígono.
Can you write Python code for it?

# # Help function to show the documentation of Shapely's Polygon
# help(Polygon)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Também podemos imprimir os pontos para ver a sua definição:
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Vamos ver como a variável do tipo LineString é mostrada no jupyter:
Can you write Python code for it?

# Visualize the line
line

print("line: \n", line, "\n")
print("line2: \n", line2, "\n")


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
Vamos ver como a variável do tipo Point é mostrada no jupyter:
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Objects.ipynb".
The first task is:
É possível recuperar atributos específicos, como o tamanho da linha e o ponto central (centróide) diretamente do objeto:
Can you write Python code for it?

# Get the lenght of the line
l_length = line.length

# Get the centroid of the line
l_centroid = line.centroid

# What type is the centroid?
centroid_type = type(l_centroid)

# Print the outputs
print("Length of our line: {0:.2f}".format(l_length))
print("Centroid of our line: ", l_centroid)
print("Type of the centroid:", centroid_type)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Agora podemos utilizar a função within() para selecionar apenas os eventos que aconteceram dentro da AIS 6.
Can you write Python code for it?

mask = data_gdf.within(ais6.geometry[0])
mask

data_gdf_ais6 = data_gdf[mask]
data_gdf_ais6


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Agora vamos mostrar somente as fronteiras das AIS e os nosso eventos de crimes.
Mas antes bora transformar os nosso dados de roubo em um GeoDataFrame.
Can you write Python code for it?

data_gdf = gpd.GeoDataFrame(data)
data_gdf.crs = ais_gdf.crs
data_gdf.head()


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Agora vamos transformar os endereços em coordenadas usando geocode() com a ferramente de busca de dados Nominatim que realiza consultas no OpenStreetMap.
Antes será necessário instalar a biblioteca geopy com o pip, para isso utilize o comando: pip install geopy
Can you write Python code for it?

# Import the geocoding tool
from geopandas.tools import geocode

# Geocode addresses with Nominatim backend
geo = geocode(data['endereco'], provider = 'nominatim', user_agent ='carlos')

geo


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Como resultado, temos um GeoDataFrame que contém nosso endereço e uma coluna 'geometry' contendo objeto Point que podemos usar para exportar os endereços para um Shapefile por exemplo.
Como os indices das duas tabelas são iguais, podemos unir facilmente.
Can you write Python code for it?

data['geometry'] = geo['geometry']
data.head()


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Existem duas colunas que podem nos ajudar a filtrar a AIS desejada, a coluna AIS e a coluna NM_AIS. Vamos utilizar a primeira por ser necessário utilizar apenas o número.
Can you write Python code for it?

ais6 = ais_gdf[ais_gdf['AIS'] == 6]
ais6.plot()
plt.show()

ais6_geometry = ais6.iloc[0].geometry
ais6_geometry

type(ais6)


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Notas sobre a ferramenta Nominatim
Nominatim funciona relativamente bem se você tiver endereços bem definidos e bem conhecidos, como os que usamos neste tutorial. No entanto, em alguns casos, talvez você não tenha endereços bem definidos e você pode ter, por exemplo, apenas o nome de um shopping ou uma lanchonete. Nesses casos, a Nominatim pode não fornecer resultados tão bons e, porém você pode utilizar outras APIs como o Google Geocoding API (V3).
2. Operações entre geometrias
Descobrir se um certo ponto está localizado dentro ou fora de uma área,
ou descobrir se uma linha cruza com outra linha ou polígono são
operações geoespaciais fundamentais que são frequentemente usadas, e selecionar
dados baseados na localização. Tais consultas espaciais são uma das primeiras etapas do fluxo de trabalho ao fazer análise espacial.
2.1 Como verificar se o ponto está dentro de um polígono?
Computacionalmente, detectar se um ponto está dentro de um polígono é mais comumente feito utilizando uma fórmula específica chamada algoritmo Ray Casting. Em vez disso, podemos tomar
vantagem dos predicados binários de Shapely
que podem avaliar as relações topológicas com os objetos.
Existem basicamente duas maneiras de conduzir essa consulta com o Shapely:

usando uma função chamada
   .within()
   que verifica se um ponto está dentro de um polígono
usando uma função chamada
   .contains ()
   que verifica se um polígono contém um ponto

Aviso: apesar de estarmos falando aqui sobre a operação de Point dentro de um Polygon, também é possível verificar se um LineString ou Polygon esta dentro de outro Polygon.

Vamos primeiro criar um polígono usando uma lista de coordenadas-tuplas e um
   par de objetos pontuais
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Perceba que apesar de possuírmos informação de endereço, não temos coordenadas dos eventos, o que dificulta qualquer tipo de análise. Para obtermos as coordenadas vamos fazer o geocoding dos endereços.
Mas antes, vamos unir todas as informações de endereço em uma coluna só chamada de endereco.
Can you write Python code for it?

data['endereco'] = data['logradouro'] + ', ' + data['localNumero'].apply(str)
data.head()


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Relembrando o endereço dos nosso dados, dois roubos aconteceram na avenida bezerra de menezes próximos ao north shopping. Sabendo que a AIS que contém o shopping é a de número 6, vamos selecionar somente os eventos de roubo dentro da AIS 6.
Primeiro vamos separar somente a geometria da AIS 6. Antes vamos visualizar os dados e verificar qual coluna pode nos ajudar nessa tarefa.
Can you write Python code for it?


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Sim, as duas operações são verdade e podemos ver isso plotando os dois objetos juntos.
Can you write Python code for it?

# Create a MultiLineString from line_a and line_b
multi_line = MultiLineString([line_a, line_b])
multi_line


I am working on the file "2020/05-geographic-information-system/Notebook_Geometric_Operations.ipynb".
The first task is:
Vamos verificar se esses pontos estão dentro do polígono
Can you write Python code for it?

# Check if p1 is within the polygon using the within function
print(p1.within(poly))

# Check if p2 is within the polygon
print(p2.within(poly))